Media Engine and Mp3 decoding

Shapyi · Post by **Shapyi** » Wed Feb 11, 2009 11:49 am

I am working with an application that uses the media engine for extra CPU power. I would like to move mp3 decoding from the main CPU and use the media engine hardware method cooleyes discovered. Is it possible to do that or are there some restrictions I should know about before I start working on it? Thanks for the help.

Torch · Post by **Torch** » Wed Feb 11, 2009 11:58 am

For simply MP3 decoding you can use the sce* MP3 functions. It will decode on the ME. There are samples in the newer SDK.

Shapyi · Post by **Shapyi** » Wed Feb 11, 2009 12:08 pm

I know, but is it possible to run my own code on the media engine and decode mp3 (say in a separate thread maybe)?

Torch · Post by **Torch** » Wed Feb 11, 2009 3:28 pm

Yes you can run your own code on the ME. Search for crazyc's sample.

Shapyi · Post by **Shapyi** » Wed Feb 11, 2009 3:32 pm

Yeah I know you could. But I know Sony's PRX sets up the media engine for decoding mp3 audio, or h.264 video. My question is, when the ME is in that state and decoding "media" can I still run my own code on it by using threads or some other method?

Torch · Post by **Torch** » Wed Feb 11, 2009 3:33 pm

Don't know about that :/
I wonder if MP3 uses the ME or the VME??

hlide · Post by **hlide** » Thu Feb 12, 2009 6:24 am

Shapyi wrote:Yeah I know you could. But I know Sony's PRX sets up the media engine for decoding mp3 audio, or h.264 video. My question is, when the ME is in that state and decoding "media" can I still run my own code on it by using threads or some other method?

Most probably the second processor for *EITHER* decoding mp3 audio, or h.264 video *OR* running your own code and not *BOTH*. In both cases, you need a processor to wait for events. Due to the fact that official developers cannot run any code on the second processor, there is very few chances for us to be able to inject our own code to run them when the mp3/h.264 decoding code is idling without messing up.

Torch · Post by **Torch** » Thu Feb 12, 2009 6:08 pm

Then what is the VME for? I was given to believe that the whole point of the VME was to hardware decode Atract and possibly other music codecs, and for applying equalizer effects.

I'm sure the ME handles the video decoding though, cos the VME is only audio. Though I have a feeling, the ME runs the code to "manage" the VME in the case of audio, but the actual decoding is on VME.

J.F. · Post by **J.F.** » Thu Feb 12, 2009 6:51 pm

More likely to be the other way around... the ME can't possibly be decoding the video - a 333 MHz MIPS cannot decode 720x480 h.264 in real-time. The VME MUST be doing most of the work. The ME is probably used for decoding the container, moving data, and colorspace conversion at most. It may also be decoding audio formats the VME doesn't understand.

hitchhikr · Post by **hitchhikr** » Thu Feb 12, 2009 10:12 pm

There's a specific hw component in the console to decode such videos, nothing to do with the VME which is only used for sound, i think.

J.F. · Post by **J.F.** » Fri Feb 13, 2009 8:46 am

hitchhikr wrote:There's a specific hw component in the console to decode such videos, nothing to do with the VME which is only used for sound, i think.

Well, looking at the block diagram, the AVC decoding is shown as a different block, but it's certainly not being decoded by the ME. It's a hardware module doing the heavy lifting. :)

Shapyi · Post by **Shapyi** » Fri Feb 13, 2009 12:51 pm

Well for my project I would only need Mp3 decoding using Sony's modules while using the media engine CPU for other operations. Anyone know if that is possible?

Torch · Post by **Torch** » Fri Feb 13, 2009 1:21 pm

The PS3 must have dedicated H264 decoding hardware too. There's no way a crappy 3.2GHz CPU that's not half as powerful as a desktop 3.2GHz CPU can decode 1080p. My old 3200+ couldn't.

Wally · Post by **Wally** » Fri Feb 13, 2009 6:07 pm

Torch wrote:The PS3 must have dedicated H264 decoding hardware too. There's no way a crappy 3.2GHz CPU that's not half as powerful as a desktop 3.2GHz CPU can decode 1080p. My old 3200+ couldn't.

Uhh its PPC.. PPC is hot damn digitty, its about 2x as powerful as your CPU xD

Torch · Post by **Torch** » Fri Feb 13, 2009 10:53 pm

Wally4000 wrote:Uhh its PPC.. PPC is hot damn digitty, its about 2x as powerful as your CPU xD

It'd probably barely manage the 5MBytes/s Bluray spec on CPU alone. No way it'd play 10+ MBps videos from the XMB if it was just CPU. It can't use the RSX either cos it doesn't support CUDA. 20MBps footage brings my C2Q @ 3.52GHz to its knees, decoding on 3 cores.

Mihawk · Post by **Mihawk** » Sat Feb 14, 2009 4:32 am

Torch wrote:The PS3 must have dedicated H264 decoding hardware too. There's no way a crappy 3.2GHz CPU that's not half as powerful as a desktop 3.2GHz CPU can decode 1080p. My old 3200+ couldn't.

The hardware is called SPE(s) ;)

Chrighton · Post by **Chrighton** » Sat Feb 14, 2009 6:49 am

If the SPEs had a destiny, it would be signal processing - they should be able to chew through a couple (and I think I'm being conservative) concurrent 1080p sources quite easily.

J.F. · Post by **J.F.** » Sat Feb 14, 2009 7:33 am

Shapyi wrote:Well for my project I would only need Mp3 decoding using Sony's modules while using the media engine CPU for other operations. Anyone know if that is possible?

Probably not. The ME is almost certainly used to pass data to/from the VME/AVC decoders, among other tasks. It's clear that Sony loads ME controlling software as part of the codecs, and since the ME is doing the controlling, that's why it's called an ME codec. The ME might not be doing that actual decoding itself, but it's in control of the whole process.

The code we use to run stuff on the ME was based on parts of the code used by Sony to start their stuff on the ME as part of those codecs, so it's a dead certainty they interfere with each other. One or the other, not both.

Shapyi · Post by **Shapyi** » Sat Feb 14, 2009 11:01 am

J.F. wrote:
Shapyi wrote:Well for my project I would only need Mp3 decoding using Sony's modules while using the media engine CPU for other operations. Anyone know if that is possible?
Probably not. The ME is almost certainly used to pass data to/from the VME/AVC decoders, among other tasks. It's clear that Sony loads ME controlling software as part of the codecs, and since the ME is doing the controlling, that's why it's called an ME codec. The ME might not be doing that actual decoding itself, but it's in control of the whole process.

The code we use to run stuff on the ME was based on parts of the code used by Sony to start their stuff on the ME as part of those codecs, so it's a dead certainty they interfere with each other. One or the other, not both.

Makes sense. I haven't been able to get anything running, even as a sample.

So the ME actually, controls the VME... does that mean it might be possible to use the VME? Has anyone been able to access that piece of hardware for use? I know Sony can do as they wish with the VME, but has anyone been able to do anything with it in terms of homebrew?

I am trying to write a media player for PSP and I would like to off load as much to hardware as possible. Sony's modules are too limited in terms of MPEG4 and AVC decoding (resolution restrictions, etc). I read that the VME is a FPGA. I was wondering if it would be possible to access it in order to offload audio decoding, color space conversion and idct in order to save CPU cycles.

Thank you for the answer and keeping this on topic.

crazyc · Post by **crazyc** » Sat Feb 14, 2009 1:16 pm

So the ME actually, controls the VME... does that mean it might be possible to use the VME? Has anyone been able to access that piece of hardware for use? I know Sony can do as they wish with the VME, but has anyone been able to do anything with it in terms of homebrew?

AFAIK, no one knows what the VME or the AVC really are. There's probably a DCT accelerator in there somewhere, but how to use it and what else is in there, Sony only knows.

I read that the VME is a FPGA

I don't think there's any evidence for that.

cheriff · Post by **cheriff** » Sat Feb 14, 2009 3:03 pm

Maybe not an fpga as it were, but it does seem to be reconfigurable under software control in perhaps a similar manner.:

Some slides:
http://www.extremetech.com/slideshow/0, ... 950,00.asp

And some interesting reading from the horse's mouth, on the vme:
http://www.sony.net/Products/SC-HP/cx_n ... view42.pdf

Other than that, there doesn't seem to be a lot out there, but I can guess that reversing the configuration bitstream format from a Sony module to the point of being able to emit custom ones ... would be difficult.

J.F. · Post by **J.F.** » Sun Feb 15, 2009 8:23 am

Basically, there's only two lines of work here: using Sony's "codecs" for decoding video and audio, and using the ME to run code. The former isn't low-level as it uses the Sony interface from user mode. The latter is COMPLETELY low-level and requires a kernel mode prx for user mode homebrew.

cualquiercosa327 · Post by **cualquiercosa327** » Mon Feb 01, 2010 5:11 am

hi.I know some parts of this has been posted before in several post but I dont know where asked it and if it could be an silly question.(correct me if i mistake,please)

First i know we can use me as other cpu (thanks to several people of this forum,using j.f / crazyc prx/samples as StrmnNrmn has done on daedalus.)
I read on a older post StrmnNrmn created a 'job manager' on top mediaengine.prx, which coordinates batches of work between the main CPU and the ME.
he uses J.F.'s MediaEngine.prx ,(i also have read it is not "finished" because some problem as not being able to stop the unit and other things)

The other way we can use is as cooleyes demos/programs using sony'prx and being able to decoder multimedia with hardware (they are many doubts how internally it works,how vme works...)

as PspPet said in a older post (2005):
"It appears the VME software is tied into the ME (Media Engine).

Suggestions/speculation:
If interested in the ME software, take a look at the "mebooter.prx" and "mebooter_umdvideo.prx" system files. Both of these files contain a GZIP copy of the ME software at the end (the mebooter.prx version is bigger, over 800KB).
(ie. the ME code is unziped, saved to memory, and the ME is reset).
You can disassemble this with a MIPS disassembler (the ME is the 2nd MIPS CPU). About half of it is MIPS code. The other half is data, some of which looks like it *might* be FPGA initialization data for the VME. It could also be general data tables (for the ME CPU), and/or code/data for the AVC (MPEG4 decoder).
A lot of reverse engineering required to figure it out."

I imagine the fist part of code of mebooter.prx initialized the ME and pass the data/instruction to vme and load/wait other prx with codec (aac,mp3..) is loaded.

could be posible create other 'job manager' into J.F.'s MediaEngine.prx
and use some parts of this disasmed code also ,making a "multitasking" mediaengine.prx able to use both extra cpu and as sony uses as vme starter/coordinator.

I am not talking to trying to understand the vme code,only reuse (imagine modificating something) the sony's dissasmed code .

dani

SilverSpring · Post by **SilverSpring** » Mon Feb 01, 2010 2:05 pm

The ME firmware was only embedded into those prxs upto 2.00. From 2.50 onwards they are stored in /kd/resource/me_xxx.img

No one has really documented the ME firmware extensively. It's quite complicated and relies heavily upon some type of MMIO at 0x4000000 (which on the main CPU is mapped to the EDRAM of the GE to be used as VRAM). This MMIO on the ME hasn't been documented either.

So at the moment, no one really knows how the VME works or how it's used.

forums.ps2dev.org

Media Engine and Mp3 decoding

Media Engine and Mp3 decoding

ME/VME...could this be possible?