apascan - Handy tool for Linux

Discuss the development of software, tools, libraries and anything else that helps make ps2dev happen.

Moderators: cheriff, Herben

Post Reply
User avatar
Chewi
Posts: 104
Joined: Sun Nov 26, 2006 12:49 pm
Location: Perth, Scotland
Contact:

apascan - Handy tool for Linux

Post by Chewi »

I've written a program for Linux that scans APA partitioned hard drives and uses Linux's device-mapper to create block devices for each partition. This means you can plug the hard drive into your PC and work directly with any Linux partitions you may have on the drive. No more fudging around with sector offsets! You can even mount installed games as ISOs, though I'm not sure whether that's really very useful.

This program could also potentially be used to boot the PS2 into Linux from an APA partition. It runs in userspace but an initrd/initramfs could do the necessary work before the system boots. I have Linux 2.6 in mind here but MontaVista's APA code is adequate for 2.4. I did try and update that code for 2.6 so that everything could be done in kernelspace but the non-contiguous partitions made it too messy.

Anyway, it's actually not 100% finished yet, it doesn't seem to get the offsets for the additional partition segments quite right but it's nearly there and you can still see it in action.

Code: Select all

root@rhapsody # ./apascan /dev/hdb
apascan v0.1 - Copyright &#40;c&#41; 2008 James Le Cuirot <[email protected]>
Distributed under the GNU General Public License v2 with no warranty.

Scanning /dev/hdb...
Found partition "__mbr" with 1 parts.
Mapping as "hdb-apa-__mbr".
Found partition "PP.HDL.Gunbird Special Edition" with 3 parts.
Mapping as "hdb-apa-PP.HDL.Gunbird_Special_Edition".
2 partitions mapped.
root@rhapsody # ls /dev/mapper
control  hdb-apa-__mbr  hdb-apa-PP.HDL.Gunbird_Special_Edition
root@rhapsody # mount /dev/mapper/hdb-apa-PP.HDL.Gunbird_Special_Edition iso
root@rhapsody # ls iso
bootmenu.bin  inaf.bin  modules  sles_530.21  system.cnf  wallpapr.bin
I'll upload it to my Gentoo PS2 wiki when it's ready.
Mega Man
Posts: 260
Joined: Sat Jun 18, 2005 3:14 am
Contact:

Post by Mega Man »

This is very interesting. Please post a link when your are finished.

As I understand you, I can also use it to install (copy) Linux on the hard disc without a PS2 (as long I have created the partition on the PS2).
User avatar
Chewi
Posts: 104
Joined: Sun Nov 26, 2006 12:49 pm
Location: Perth, Scotland
Contact:

Post by Chewi »

That is partially correct, you could use ps2fdisk on your machine and save yourself the bother of creating the partitions on the PS2. :)

It's ready now. It seems to work well, the only problem I've had is that the DMS HDD Format Tool appears to create partition tables that not even hdl_dump, let alone apascan, can read. uLaunchELF can read them though. Maybe this isn't really a problem but I expected it to work and I want to know what's up with it.

http://gps2.aura-online.co.uk/trac/gent ... ki/apascan
User avatar
RashivaX
Posts: 4
Joined: Sat Mar 15, 2008 2:37 pm
Contact:

Any progress on DMS HDD formatted partitions?

Post by RashivaX »

I currently have a 320 GB HD installed in my PS2 and am wanting to use a bit of the space for media, such as music, etc... I created two extra partitions with DMS, but network transfers are incredibly slow. If I want to upload mass amounts of media, it will take a hair ripping amount of time.

I need to be able to mount the partitions on one of my PC's... one, my primary, is running Ubuntu Linux... so this works nicely for apascan :) but, again, the problem: DMS formatted... so, if not the ability to handle these partitions through apascan, and this gets into different territory, is there another app I can use to create partitions which would be fully compatible/readable on both the PC and the PS2? You mention ps2fdisk... not used this, but could I use this without losing current data--- just dealing with unallocated space?

BTW, this is another question probably better in another thread, but I will ask it anyway, since it's related... I have created one of the DMS partitions as 60GB... DMS does not show the size correctly... makes me slightly nervous... especially when thinking about how DMS is doing something somewhat mysterious, since you were having the issue with the partitions showing... so, am fearing other data might be overwritten.

Anyway, help on the main point at least, is much appreciated... thanks...
User avatar
RashivaX
Posts: 4
Joined: Sat Mar 15, 2008 2:37 pm
Contact:

apascan compile issues

Post by RashivaX »

Having another problem now.. that is with compiling apascan.

Am using Ubuntu 7.10 amd64.

First got this error about libdevmapper:

Code: Select all

checking libdevmapper.h usability... no
checking libdevmapper.h presence... no
checking for libdevmapper.h... no
configure&#58; error&#58; libdevmapper not found. Get it from http&#58;//sourceware.org/dm.
trying to install libdevmapper gave this:

Code: Select all

&#58;~/DL/apascan-0.1$ sudo apt-get install libdevmapper
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting libdevmapper1.02.1 instead of libdevmapper
libdevmapper1.02.1 is already the newest version.&#91;
I then installed libdevmapper-dev and also did a:

Code: Select all

sudo modprobe dm-crypt
Which I am uncertain about... saw this in a thread on ubuntuforums in regards to an issue someone was having with crypto--- my knowledge is VERY limited here :( But, thought I should mention all I can...

This seems to have gotten through the ./configure fine.. but when I go to make, I get:

Code: Select all

gcc -DHAVE_CONFIG_H -I.     -g -O2 -MT apascan.o -MD -MP -MF .deps/apascan.Tpo -c -o apascan.o apascan.c
In file included from apascan.c&#58;26&#58;
/usr/include/sys/types.h&#58;46&#58; error&#58; conflicting types for ‘loff_t’
/usr/include/linux/types.h&#58;30&#58; error&#58; previous declaration of ‘loff_t’ was here
/usr/include/sys/types.h&#58;62&#58; error&#58; conflicting types for ‘dev_t’
/usr/include/linux/types.h&#58;13&#58; error&#58; previous declaration of ‘dev_t’ was here
In file included from /usr/include/sys/types.h&#58;133,
                 from apascan.c&#58;26&#58;
/usr/include/time.h&#58;105&#58; error&#58; conflicting types for ‘timer_t’
/usr/include/linux/types.h&#58;22&#58; error&#58; previous declaration of ‘timer_t’ was here
In file included from apascan.c&#58;26&#58;
/usr/include/sys/types.h&#58;198&#58; error&#58; conflicting types for ‘int64_t’
/usr/include/linux/types.h&#58;98&#58; error&#58; previous declaration of ‘int64_t’ was here
/usr/include/sys/types.h&#58;204&#58; error&#58; conflicting types for ‘u_int64_t’
/usr/include/linux/types.h&#58;97&#58; error&#58; previous declaration of ‘u_int64_t’ was here
In file included from /usr/include/sys/types.h&#58;220,
                 from apascan.c&#58;26&#58;
/usr/include/sys/select.h&#58;78&#58; error&#58; conflicting types for ‘fd_set’
/usr/include/linux/types.h&#58;12&#58; error&#58; previous declaration of ‘fd_set’ was here
In file included from apascan.c&#58;26&#58;
/usr/include/sys/types.h&#58;235&#58; error&#58; conflicting types for ‘blkcnt_t’
/usr/include/linux/types.h&#58;124&#58; error&#58; previous declaration of ‘blkcnt_t’ was here
In file included from /usr/include/inttypes.h&#58;28,
                 from /usr/include/libdevmapper.h&#58;19,
                 from apascan.c&#58;35&#58;
/usr/include/stdint.h&#58;56&#58; error&#58; conflicting types for ‘uint64_t’
/usr/include/linux/types.h&#58;96&#58; error&#58; previous declaration of ‘uint64_t’ was here
make&#91;1&#93;&#58; *** &#91;apascan.o&#93; Error 1
Any ideas?
User avatar
Chewi
Posts: 104
Joined: Sun Nov 26, 2006 12:49 pm
Location: Perth, Scotland
Contact:

Post by Chewi »

First of all, as it says in the README, apascan won't handle drives larger than 127GB. The official APA format doesn't support this and larger sizes were only achieved through the 48-bit LBA hack and the ToxicOS APAEXT format. I'd like to support them but I don't have any drives larger than 120GB.

You were right to install libdevmapper and libdevmapper-dev. I don't think dm-crypt is necessary and you wouldn't need to modprobe anything until after it's built anyway.

The header conflict you're getting may be a Ubuntu-specific issue and it appears to be due to a clash between x86 and amd64 headers. I don't know how Ubuntu handles multilib but Gentoo does it by replacing the usual /usr/include/sys/types.h with the following and does the same for many other header files.

Code: Select all

/* Autogenerated by create_ml_includes&#40;&#41; in multilib.eclass */

#ifdef __i386__
# include <gentoo-multilib/x86/sys/types.h>
#endif /* __i386__ */

#ifdef __x86_64__
# include <gentoo-multilib/amd64/sys/types.h>
#endif /* __x86_64__ */
That picks out the appropriate file for the architecture you're building for. I don't have access to a Ubuntu amd64 system, only a Debian amd64 one. I haven't tried building on it (it's not really my machine) but /usr/include/sys/types.h has the usual contents so I'm not sure how they handle multilib. If I were to guess, I'd say that yours is the amd64 version, as it should be, and it's actually the headers in /usr/include/linux that are wrong. Maybe you installed the wrong kernel headers somehow?
User avatar
RashivaX
Posts: 4
Joined: Sat Mar 15, 2008 2:37 pm
Contact:

Post by RashivaX »

Should have replied back before...just remembered, but not much that can be done since there is not support currently for drives larger than 127GB.

My Kernel is generic. linux-headers-2.6.22.14-generic. I haven't had this amd64 for very long and did a fresh install of Ubuntu 7.10 amd64 bit version... this is what was installed. Could it be that it is generic that it is causing problems?

Again, knowledge is limited. Only been a dedicated Linux user for about a year and no good books, just learn as I go along from forums and trial and error :)

I would give you access to this machine to compile on if not for distance... eh well...
User avatar
Chewi
Posts: 104
Joined: Sun Nov 26, 2006 12:49 pm
Location: Perth, Scotland
Contact:

Post by Chewi »

That package appears to be the correct one. I did a little digging and found a similar issue. Try moving the "#include <sys/types.h>" line to the top of the list of includes in apascan.c. Let me know if this works.
User avatar
RashivaX
Posts: 4
Joined: Sat Mar 15, 2008 2:37 pm
Contact:

Post by RashivaX »

Ok, done... appears to have compiled and installed properly. I get the output text for apascan :) But, I cannot test it further ATM... I do have a 60GB drive that I could try within a day or so.

This probably won't make much sense to me anyway, but why was that the problem?
User avatar
Chewi
Posts: 104
Joined: Sun Nov 26, 2006 12:49 pm
Location: Perth, Scotland
Contact:

Post by Chewi »

The suggestion came from this bug. I've just had a look through the files to see what's going on.

asm/byteorder.h includes linux/byteorder/little_endian.h and that includes linux/types.h. This defines types such as loff_t but only when __KERNEL_STRICT_NAMES is false. At this point, it is false so they are defined. loff_t is then redefined as a conflicting type later on in sys/types.h.

sys/types.h includes features.h and this sets __KERNEL_STRICT_NAMES to true. By including sys/types.h first and having this set to true, loff_t does not get redefined in linux/types.h later on.

Why this happened to you but not me and whether this was really my fault or not is a mystery! Thanks for helping me fix this. I'll put out a new version soon.
cosmito
Posts: 307
Joined: Sun Mar 04, 2007 4:26 am
Location: Portugal
Contact:

Post by cosmito »

Hi Chewi,

It seems gps2.aura-online.co.uk is not online... Definitely off? Any chance of uploading the apascan to somewhere else?
User avatar
Chewi
Posts: 104
Joined: Sun Nov 26, 2006 12:49 pm
Location: Perth, Scotland
Contact:

Post by Chewi »

Apologies for not replying, I didn't see this message until now. Since I've largely given up on PS2 development and have moved from SVN to GIT, I wasn't in a hurry to get that site back up. I have now created a repository for apascan on github instead.

http://github.com/chewi/apascan
Post Reply