Database Info

Compressed Resources (resources.arsc) Decompressor

First time poster, but long-time lurker and avid Android Developer here.
I'm putting the finishing touches on a tool that decompresses resource files (including the ARSC and any compressed XML files). It's something I sort of took interest in in my spare time, as a learning experience, and I think it would be helpful to the community. It could probably be used to make modifications to compressed layouts in a ROM, such as HTC Sense-based ROMs (decompress the resources, make edits, compress, sign...).
Anyways, I figured I'd ask first... does a tool like this already exist? If so, whatever, this was a learning experience anyways. If not, I'd like to get it out there for all of you geniuses to use.
I'd also like to know what kind of options might be good to have on this tool. Right now it's command-line-based (and might stay that way... I think a UI might be overkill). Let me know. I'll be watching!

That's great!
In which language is it written? Will you open-source it? If so, on which license?
I'm asking cause I need such tool for my Omnipatcher project and I intended to make it myself

Java. I'll probably open-source it once I clean it up enough. I mean, nothing's really a secret in there. I figured out everything I needed from the Android sources.
Brut.all said:
That's great!
In which language is it written? Will you open-source it? If so, on which license?
I'm asking cause I need such tool for my Omnipatcher project and I intended to make it myself
Click to expand...
Click to collapse

When?
When will you relase this?

Oh, good work!!!
Any news?

itanczos said:
Oh, good work!!!
Any news?
Click to expand...
Click to collapse
Sorry guys, I'm really eager to get this out, I'm just struggling to pay the bills, too. I hesitate to make promises, but it should be out sometime this month. I'm just as excited as you probably are to use it. I can't wait to see what kind of themes/mods sprout up once you all get your hands on this.

That sound cool, I was also thinking in creating such a tool or maybe just a shell script that uses aapt to get all the infos and generate an xml out of it but if you already have something in the pipe for doing this... I hope it's finished (or better said at a release stage) soon.

rac2030 said:
That sound cool, I was also thinking in creating such a tool or maybe just a shell script that uses aapt to get all the infos and generate an xml out of it but if you already have something in the pipe for doing this... I hope it's finished (or better said at a release stage) soon.
Click to expand...
Click to collapse
Doesn't aapt only compile the resources, and not the other way around? I didn't think aapt gave us all the information we needed to go back to the original XML.

binarybulge said:
Doesn't aapt only compile the resources, and not the other way around? I didn't think aapt gave us all the information we needed to go back to the original XML.
Click to expand...
Click to collapse
It has dump command and output looks like full XML data just in different (easy to parse) format:
Code:
N: android=http://schemas.android.com/apk/res/android
E: manifest (line=44)
A: android:sharedUserId(0x0101000b)="com.google.android.apps.maps" (Raw: "com.google.android.apps.maps")
A: android:versionCode(0x0101021b)=(type 0x10)0xcf6
A: android:versionName(0x0101021c)="3.3.1" (Raw: "3.3.1")
A: package="com.google.android.apps.maps" (Raw: "com.google.android.apps.maps")
E: uses-sdk (line=54)
A: android:minSdkVersion(0x0101020c)=(type 0x10)0x4
E: uses-permission (line=58)
A: android:name(0x01010003)="android.permission.CALL_PHONE" (Raw: "android.permission.CALL_PHONE")

binarybulge said:
Doesn't aapt only compile the resources, and not the other way around? I didn't think aapt gave us all the information we needed to go back to the original XML.
Click to expand...
Click to collapse
Code:
aapt dump xmltree xxx.apk AndroidManifest.xml
This does output some sort of xml like output... at least as far I have analyzed the output, it should be possible with some parsing code to recover or better said reconstruct a working xml ;-)
Of course, just implementing a complete encoder/decoder would be a nicer solution and as you said, theoretically all the needed framework stuff is on git so it wouldn't be hard to implement it if you have time... I though that this was what you have done or not?

rac2030 said:
Code:
aapt dump xmltree xxx.apk AndroidManifest.xml
This does output some sort of xml like output... at least as far I have analyzed the output, it should be possible with some parsing code to recover or better said reconstruct a working xml ;-)
Of course, just implementing a complete encoder/decoder would be a nicer solution and as you said, theoretically all the needed framework stuff is on git so it wouldn't be hard to implement it if you have time... I though that this was what you have done or not?
Click to expand...
Click to collapse
Haha, yeah it is what I have done. You guys just kind of worried me a little making me think I was reinventing the wheel.
aapt would have been one approach, but I'm still not sure it covers all bases. For example, the strings.xml, arrays.xml, etc files. Those obviously aren't handled the same as layout files. Their contents get compressed into the arsc file. I'm also handling some more complex cases, such as one package referencing drawables from another package.
My goal of course is to restore all input XML, including things like strings.xml, and all of those in various configuration-specific folders (orientation, locales, screen sizes...).

is there any public source of this Compressed Resources (resources.arsc) Decompressor?
i'd like to test it!

Hello Binarybulge!
News?

Is this dead or what?

I'm working on such tool on my own, have managed to decode XMLs (using Android source, not parsing aapt dumps) and now I know, what binarybulge was talking about:
binarybulge said:
aapt would have been one approach, but I'm still not sure it covers all bases. For example, the strings.xml, arrays.xml, etc files. Those obviously aren't handled the same as layout files. Their contents get compressed into the arsc file. I'm also handling some more complex cases, such as one package referencing drawables from another package.
My goal of course is to restore all input XML, including things like strings.xml, and all of those in various configuration-specific folders (orientation, locales, screen sizes...).
Click to expand...
Click to collapse
binarybulge: please, let me know, whether you have quit, don't have time, died or what? Currently I'm working on decoding @ids and /res/values/ and I don't want to reinvent the wheel, if you have done this so far and just don't have time to continue your work.

I'm interested in pitching in. I want an easy tool for decoding a binary .xml file, edit it including adding new elements and then convert it back to binary xml.
I'm pretty familiar with Android low level stuff. One example of my work:
http://forum.xda-developers.com/showthread.php?p=5475283
If I can help in any way, let me know. I don't want to reinvent the wheel either.

jonasl said:
I'm interested in pitching in. I want an easy tool for decoding a binary .xml file, edit it including adding new elements and then convert it back to binary xml.
I'm pretty familiar with Android low level stuff. One example of my work:
http://forum.xda-developers.com/showthread.php?p=5475283
If I can help in any way, let me know. I don't want to reinvent the wheel either.
Click to expand...
Click to collapse
Just for curiosity: how did you do it? Hex edited xml's and resources.arsc?
I'm still working on this tool and have made some progress

Everything that's been done on the keyboard linked above has been done in code. You of all people need no introduction to smail/baksmali
I've rewritten the configuration system (HTC's settings provider is missing in non sense roms), rewritten the parts that interfaces with google voice recognition service and some other tweaks, but it's all code mods.
To fix some remaining issues I must edit xml layouts. Just changing some color code etc. is doable in any hex editor, but adding and removing elements and attributes is kind of hard. I'm stuck at this point and was looking for a tool to convert own xml to binary xml. Since I didn't find such tool I was thinking about creating one and ran into this thread...

I've just successfully and fully automatically decoded all resources for simple HelloWorld apk, then edited them, packaged again using aapt and run on a device It's early alpha and is unusable for now cause it still doesn't support many types of resources, but I have a proof of concept, that it is possible to repackage resources

Brut.all said:
I've just successfully and fully automatically decoded all resources for simple HelloWorld apk, then edited them, packaged again using aapt and run on a device It's early alpha and is unusable for now cause it still doesn't support many types of resources, but I have a proof of concept, that it is possible to repackage resources
Click to expand...
Click to collapse
Yay! Cool! Waiting for release!
Greets!

[Release/Dev]Automatic Boot Logo Creator/ApkManager replacement

So first off, the release:
I've created a program (technically a script) written in Python that does all of the tedious work of making a boot logo and getting it on your Droid X or Droid2 phone. Specifically, it does all of the following for you:
1) Takes a 24-bit BMP of size 480px X 182px and flips the canvas horizontally
2) Makes the image the right size by removing the first 54 bytes (after the canvas flip all images are the same size so we don't need to remove the last 2)
3) Reverses all the bytes and writes the image out to a logo file (logo.bin)
4) Puts your created logo into the TBH Logo Replacer flashable zip file
5) Pushes the zip to your SD card
This tool should give anyone who has Clockwork recovery and is slightly familiar with it and knows how to copy, paste, and crop in an image editor the ability to make their own boot logo.
code.google.com/p/android-customization-autotool/downloads/detail?name=AutoBootLogoCreatorV1.1.zip
The project in development:
As you can probably tell from the link above, I'm also working on a tool that I have dubbed the Android Customization Autotool. When completed, it will have all the same functionality as ApkManager, and more. For the most part, I've been using ApkManager and it works fairly well.
However, there are some things that I feel are missing and have added in my tool. Similar to ApkManager, my tool isn't some new method of doing something or some type of breakthrough. It merely takes existing tools that are out there (and some that are not) and packages them up into an "all-in-one" type tool that is easy and efficient to use for beginners and experts alike.
What I'm aiming to do with the tool is quite simple. I'd just taking the significant amount of scattered knowledge on various forums regarding hacks and customizations and automating the whole thing so that people who don't have the know how, or aren't confident in their know-how can have all of the same customization opportunities that the rest of us who know what we're doing have. And for those who do know what they're doing, the tool will save a lot of time and effort.
The project is located at:
code.google.com/p/android-customization-autotool/
If anyone is interested in helping out (note it's in Python) just let me know. I'm also extremely open to any suggestions that people have about what should be supported. There are still a ton of customizations out there that I haven't seen any tools for and it would be nice to cover everything.
For those of you who would like to continue using ApkManager (since right now it's the better choice), I've been designing my tool in a modular way where any functionality that is provided can be forked off into it's own stand-alone application. And since the project is open-source, you can add or strip away as much or as little as you want. If you do make something cool though, please let me know because I'd love to add it to the tool (credit is always due of course).

thank you for sharing

Thank You...

If you guys run into any problems with the code, please let me know via the Issues page at code.google.com/p/android-customization-autotool/issues/list and I'll do my best to fix them. Still working on the all-in-one, but I've made an Update Zip auto creator that generates the META-INF folder, creates a .zip and signs it. More on that later today, sleep time =D

Thank you!!

[TOOL][BETA] ID Wizard

Hey
Aplologies if this is in the wrong place! Please move if necessary.
So a while ago I tried Untermenschs Resource Mapper.
It worked ok but it would sometimes miss ID's and sometimes would find ID's incorrectly where the same name existed for say a string and an id.
I was porting lockscreens through SGS2 firmwares, had excel 2003 installed and a bit of VBA knowledge so thought I'd give it a go myself.
It works on the same principal. Mod your framework-res.apk with the images, strings,ids,layouts etc etc. Compile and get your new public.xml.
I used this on files I had ported code across into from android.policy.jar and framework.jar.
When prompted, give the tool your old roms public.xml and the new one. Show it where the files are that you have ported the code across into and it will look up the IDs names and types in the old public and then find the new ID in the new public and change the file accordingley. It comments that line with the type and name for easy identification in the same way untermenschs tool did but includes the type as well.
IT OVERWRITES THE FILES, SO MAKE SURE THEY'RE NOT YOUR ONLY COPY, JUST IN CASE!
Excel 2003 is needed as I only realised afterwards that they removed filesearch from excel 2007 inwards. I will redo that part of the routine.
The code is scrappy and Im certainly no expert when it comes to VBA but this has worked 100% in the last 2 ports we have tried. The 10 lockscreens for SGS2 KL1 and 11 for SGS1 JW1.
Please someone try it and let me know how it goes. Would be nice to see if its working good for others or failing miserably.
If it saves someone as much time as it saved me in the last port it will be worth posting
Link removed for updating...

Any chances of getting the link soon ?

[DEV][M10] Decompiling M10 (Sense) images

The problem
Since HTC introduced Sense 3.5, themers faced a huge problem. The previously used software "M10Tools" wouldn't work with the new version of Sense.
Flemmard and me tried countless hours decoding the new image format, but without any success. The new image format is totally diferent to anything else previously seen.
I made this thread to search for help from all the awesome devs on XDA, hoping that we might find one who can help.
The history
Let me start this with some introduction to the m10 format itself.
The images I am talking about are parts of one big file - the m10 file. We usually have multiple images per m10 file, but the number doesn't really matter.
Together with the raw image data we get a set of meta information. We are not exactly sure what the values mean, but we can guess the meaning from the history of the old, decodable images.
We used to have information like width, height, payload of the image and an integer indicating what kind of image type we have. We know the actual image type for a few of these intergers, but with Sense 3.5, 3.6 and 4.0 HTC added at least two new types.
The facts
We don't have any hard facts for these image types but looking at the "old" image types, we can guess a few things:
The images are in a format the GPU can render directly (Like s3tc, ATC, QTC, etc) (At least this used to be the case, might have changed)
Images are most likely compressed. The ratio between assumed size (based on meta data) and the actual data size indicates some heavy compression. The data itself obviously looks compressed too.
There are no headers or any other help. It is just raw data.
We don't know exactly how the decoded images actually look like, so we can't say what the images display. However, due to latest archievements we "might" know this for images from Sense 3.5 and 3.6 if needed.
The handling software side is all in a few libs and NOT in smali / java, so we can't look for stuff there, however we have the libs, so if someone is pro with assembler he might find out something
I will provide a download which contains several chunks of image data and the according meta data.
If you consider working on this, please do not refrain from thinking about super simple solutions, we worked so long on this that we might be totally confused.
One thing though, this might sound arrogant, but this here really is only for people who have some decent knowledge about file formats, image compression or OpenGL.
The image types
Here is a list of image type we already know ( remember, we don't know where the numbers come from, might be some enum in native code or so)
Type 4: Raw RGB
Type 6: Raw RGBA (still used rather often)
Type 8: ATC RGB (doesn't seem to be used at all anymore)
Type 9: ATC RGBA Explicit (doesn't seem to be used at all anymore)
As you can see we got types WITH and WITHOUT alpha encoding.
Here is the list of UNKNOWN formats:
Type 13 (used way less than type 14, so maybe no alpha?)
Type 14 (this is the most used type, so I assume this one supports alpha encoding)
When thinking about what the data might be, don't throw away crazy ideas like "The data is S3TC /ATC /whatever but compressed again by some 'normal' compression algorithm". Maybe they just replaced type 8 and 9 with an additional compression on top of these types.
The meta data
Okay, so now lets talk about the meta data we get together with the actual data:
We get 4 more or less known chunks of information per image (plus a few unknown things)
Image type (described earlier) (Example: 6)
Image width (Example: 98)
Image height (Example: 78)
A more complex value containing multiple values at once.
Example: "98:78:0:30576"
We used to know the meaning of three of these values. However we are not sure for the new images. Lets explain the old meaning first:
98: Width, same value as the value above
78: Height, same value as the value above
0: It's always 0, we have no idea what it means, but since it's static we didn't care
30576: this used to be the data size. This image has a resolution of 98*78 = 7644 pixels. With a data size of 30576 that means we got 4bytes per pixel.
Lets take a look at the new images now. We still get the same information, however the meaning seems to have changed a bit:
Image type (described earlier) (Example: 14)
Image width (Example: 997)
Image height (Example: 235)
A more complex value containing multiple values at once.
Example: "1000:236:0:118000"
This is the assumed new meaning
1000: Width, but rounded up to a multiple of 4
236: Height, but rounded up to a multiple of 4
0: It's always 0, we have no idea what it means, but since it's static we didn't care
118000: this value is now exactly half of the rounded resolution (1000 * 236 / 2 = 118000)
This would mean only half a byte per pixel. One big problem here: the actual data size does not match this value at all!
The data is way smaller than this value, which indicates that it got compressed a second time
Now lets talk about some very important piece of information: HTC uses the SAME image formats on BOTH a Tegra 3 and a Qualcomm Snapdragon S4.
This obviously means that both Tegra and Snapdragon need to be able to handle this. However, also keep in mind that HTC bought S3 graphics and thefore might got some advantages here.
You can find a statistic on the used formats in the download, it's an Excel sheet with two diagrams showing the usage.
Now this was a long post, I hope someone is still reading this and might have some ideas about what's going on here.
Feel free to ask any questions concerning this.
I am also available in #virtuousrom on Freenode, per PM here or via email: diamondback [at] virtuousrom [dot] com
Download:
The download contains a bunch of unknown images of types 13 and 14 together with their meta data (like explained above)
Download image pack

Solution
After some digging I estimated that the data is compressed with fastlz [0]. Also so if you decompress it, you get exactly Width*Height bytes of data. I dont know the format this data is in, but i guess its the same the uncompressed data (type 8 or 9 or so?) was. Maybe someone could check up on that.
[0] http://fastlz.org/

onlyolli said:
After some digging I estimated that the data is compressed with fastlz [0]. Also so if you decompress it, you get exactly Width*Height bytes of data. I dont know the format this data is in, but i guess its the same the uncompressed data (type 8 or 9 or so?) was. Maybe someone could check up on that.
[0] http://fastlz.org/
Click to expand...
Click to collapse
You are indeed right. We actually found the same a few hours ago What a weird conincidence... :victory:
Type 4 and 6 are changed, they are zipped now too. Which actually breaks backwards compatibility with older Sense versions...
Inside of the zipped data are ETC images, which also explains how they can use the same on S4 and Tegra 3.
The type 14 actually contains TWO images, both ETC. Since ETC doesn't support alpha one is the image and one is an alpha mask...
Funny trick HTC!

Extract UI strings from firmware / device

Hi.
Before I start with my question, just a little background. I work in a user guide development firm, and mainly work on Galaxy devices.
One of the most time-consuming process in my work is to match all UI strings (app names, menu text, labels, etc) of the actual device with the user guide.
This is currently being done with human power, with a staff looking at the user guide, check the device if this is correct, and annotate the draft PDF if the UI strings don't match.
If this was for just one language, it's doable. But with 40 or so languages (including Arabic, Cyrillic, Chinese), it definitely makes me want to puke.
This is a very tiring, eye-straining work that I'm trying to resolve, for everyone's sake.
I tried decompressing the Galaxy firmware myself, but the XML data is encoded into binary(for what reason I have no idea), and is not readable.
So now I'm turning to the masters and hope for any luck.
What I would like to know is ...
Hack the Galaxy firmware(md5), and extract UI strings for all language and save in spreadsheet or something, or
Mirror the device's screen on the desktop, copy the desired UI string, and paste it to a desktop application(e.g. Adobe Acrobat).
If the first option is possible, then I can utilize the data for some sort of automation, and would be the best.
If the second option is possible, then I would no longer have to type all kinds of foreign characters(this is also a very time-consuming work), and make some progress in timeline.
If all options are not possible, and there's absolutely no way of automating this process, then well... I guess I'll go see the eye doctor more often than now.
Any ideas or helps would be great.
Thank you.