Related
WARNING: Following these instructions may brick your phone, void your warranty and kill your dog. You don't want your dog to die do you?
Once I got root access on my G1, I've been messing around with trying to build reflash the recovery partition. That project is still in progress, but I have learned a bit about how to flash the various partitions on the G1.
First things first, you have to have root access. See this thread.
There are 6 mtd devices or partitions on the G1, mtd0-mtd5. They are located at /dev/mtd. You can use the /system/bin/flash_image tool to flash an image to any of these.
The syntax for the flash_image tool is:
Code:
# flash_image
usage: flash_image partition file.img
#
You can see a list of partition names and which device they are associated with by doing a "cat /proc/mtd".
Code:
#cat /proc/mtd
dev: size erasesize name
mtd0: 00040000 00020000 "misc"
mtd1: 00500000 00020000 "recovery"
mtd2: 00280000 00020000 "boot"
mtd3: 04380000 00020000 "system"
mtd4: 04380000 00020000 "cache"
mtd5: 04ac0000 00020000 "userdata"
#
These should all be self explanatory, except maybe "misc", which just appears to have a few string values.. Not really sure what it's used for..
Before you do any erasing or writing, it's a "really good idea" (tm) to make backups of each of these. Even if you don't plan on writing to them. I had accidentally erased the bootloader partition (typed mtd0 instead of mtd1), which I'm fairly sure would have bricked my phone if I had tried to reboot it. Ugh! Luckily, I had created a backup earlier, so I was able to restore it. (And then was nervous as heck when I tried to reboot it... "Please boot up! Please boot up!")
To create the backups:
Code:
# cat /dev/mtd/mtd0 > /sdcard/mtd0.img
# cat /dev/mtd/mtd1 > /sdcard/mtd1.img
# cat /dev/mtd/mtd2 > /sdcard/mtd2.img
# cat /dev/mtd/mtd3 > /sdcard/mtd3.img
# cat /dev/mtd/mtd4 > /sdcard/mtd4.img
# cat /dev/mtd/mtd5 > /sdcard/mtd4.img
#
Now you can use flash_image to write the new image.
Code:
#flash_image recovery /system/recovery.img
#
And that's how it's done.
Update: You normally don't need to erase the flash before you write, as long as you don't corrupt the flash first, by trying to "cat" an image directly to the mtd device, like I did . If you get a lot of ECC errors when you use flash_image then you need to erase it
Code:
# cat /dev/zero > /dev/mtd/mtd1
write: No space left on device
#
On a related note, Based on my experience so far, the recovery partition is not critical to booting the G1. When I first tried this, I used flash_image to write a new image without erasing the partition first, and it corrupted it pretty good. When I tried to boot the G1 into recovery mode (power+home), it would go to the bootloader screen instead. But it would boot into normal mode just fine.
Additionally, I've verified that the device will boot into recovery mode if you screw up the boot partition (mtd2). So as long as you at least have a good recovery image *or* a good boot image, you should be able to get back in business. Just don't try to update both at the same time.
Even so.. be careful, and don't come crying to me when you brick your phone. Or if your dog dies. You don't want your dog to die do you?
Excellent work!
So if misc is the boot loader, it looks like it is roughly 266k. If you can dump it, have you tried to "open" the image and see if you can see files?
Now you have given me more ideas
readonly sdcard?
When I run: cat /dev/mtd/mtd1 > /sdcard/mtd1.img
I got this:
cannot create /sdcard/mtd1.img: read-only file system
I know I am root...
staulkor said:
So if misc is the boot loader, it looks like it is roughly 266k. If you can dump it, have you tried to "open" the image and see if you can see files?
Click to expand...
Click to collapse
Actually, now that I look at it.. I'm not positive that mtd0 is the bootloader. There's not much data there at all. Just a few strings in the beginning, then lots of nothing. Mostly all FFs, with a few blocks of 00s thrown in. It doesn't look like there's any code at all, so it can't be the bootloader.
Nice work. How about the signing? Does the image you flash have to be correctly signed this way?
blues said:
When I run: cat /dev/mtd/mtd1 > /sdcard/mtd1.img
I got this:
cannot create /sdcard/mtd1.img: read-only file system
I know I am root...
Click to expand...
Click to collapse
Do you have the sdcard mounted for USB access? If so, just unplug the usb cable and plug it back in. (don't select "mount" in the usb notification thingy that pops up)
Chainfire said:
Nice work. How about the signing? Does the image you flash have to be correctly signed this way?
Click to expand...
Click to collapse
Nope. This is a raw write directly to the flash device. The whole signing thing is only applicable to the OTA updates (or Update.zip style update).
But the other project I'm working on is to rebuild the recovery mode and disable the signature check for OTA/update.zip updates.
If you check out the SDK and build the open-source version for the phone, it builds a custom recovery-image that accepts only things signed with the testkeys - which is cool. Since the test keys are in the directory, you can easily resign the images yourself.
I know a guy who has done this now. I'll do it when I get a chance.
There are two proprietary files that you have to suck off the device in addition to the ones that the "extract_files" script in the android build pulls off - I sent in a patch to fix this but who knows if they'll apply it or not (I thnk they think that the crap in the msm7k dir will do something)
What I'm wondering is if we can write an update mode that will backup the contents of the /cache and /data to the SDCard - then erase the three partitions - then recreate /data as a huge partition and leave only 10 or 15 megs for /cache - because... well.. /cache is worthless since OTA updates aren't gonna happen to our phones anymore anyway. It would be nice to get an extra 40 megs for app storage.
JesusFreke said:
Actually, now that I look at it.. I'm not positive that mtd0 is the bootloader. There's not much data there at all. Just a few strings in the beginning, then lots of nothing. Mostly all FFs, with a few blocks of 00s thrown in. It doesn't look like there's any code at all, so it can't be the bootloader.
Click to expand...
Click to collapse
I just looked at my dumped mtd0.img and I see a few interesting strings:
T-MOB010
DeviceWarmBoot
CE Serial InUse
Debug Cable Ena
CE USB InUse
ClearAutoImage
And then a mountain of FFs, lol
You are right. I am on XP machine. So I disabled the usb storage, and it works fine
JesusFreke said:
Nope. This is a raw write directly to the flash device. The whole signing thing is only applicable to the OTA updates (or Update.zip style update).
But the other project I'm working on is to rebuild the recovery mode and disable the signature check for OTA/update.zip updates.
Click to expand...
Click to collapse
It works for me now.
RyeBrye said:
If you check out the SDK and build the open-source version for the phone, it builds a custom recovery-image that accepts only things signed with the testkeys - which is cool. Since the test keys are in the directory, you can easily resign the images yourself.
Click to expand...
Click to collapse
Yep, that's exactly what I'm doing. I'm planning on posting a recovery.img for others to use, since trying to build the thing from scratch is a pain, and takes forever.
RyeBrye said:
There are two proprietary files that you have to suck off the device in addition to the ones that the "extract_files" script in the android build pulls off - I sent in a patch to fix this but who knows if they'll apply it or not (I thnk they think that the crap in the msm7k dir will do something)
Click to expand...
Click to collapse
I assume you're talking about libaudio.so and librpc.so?
RyeBrye said:
What I'm wondering is if we can write an update mode that will backup the contents of the /cache and /data to the SDCard - then erase the three partitions - then recreate /data as a huge partition and leave only 10 or 15 megs for /cache - because... well.. /cache is worthless since OTA updates aren't gonna happen to our phones anymore anyway. It would be nice to get an extra 40 megs for app storage.
Click to expand...
Click to collapse
Good idea. Even better would be if we could put the installed apps and all data on the sdcard.. But that would probably be a harder modification than just resizing the partitions.
staulkor said:
I just looked at my dumped mtd0.img and I see a few interesting strings:
T-MOB010
DeviceWarmBoot
CE Serial InUse
Debug Cable Ena
CE USB InUse
ClearAutoImage
And then a mountain of FFs, lol
Click to expand...
Click to collapse
that partition is the config partition, standard HTC stuff.
T-MOB010 is your CID (carrier ID, spl checks this when flashing NBH), but it is only a backup copy of it, the real CID is in radio part of nand (protected!).
and the rest is just config stuff for SPL and radio. (yes HTC uses strings to set these configs - SPL reads the strings from specific nand addresses and decides what to do)
JesusFreke said:
On a related note, Based on my experience so far, the recovery partition is not critical to booting the G1. When I first tried this, I used flash_image to write a new image without erasing the partition first, and it corrupted it pretty good. When I tried to boot the G1 into recovery mode (power+home), it would go to the bootloader screen instead. But it would boot into normal mode just fine.
Click to expand...
Click to collapse
very nice stuff! have you been able to reflash a recovery.img then to fix the corruption?
and you are right about the boot order... it's : radio bootloader -> SPL (that tricolour screen is SPL mode) -> boot.img or recovery.img.
(if boot.img then the OS loads)
cmonex said:
very nice stuff! have you been able to reflash a recovery.img then to fix the corruption?
Click to expand...
Click to collapse
Yep. I screwed up the recovery partition and rebooted, and wasn't able to boot into recovery mode. It just went into SPL mode when I tried. Then I normal booted and re-flashed with the original recovery.img and rebooted, and was able to boot into recovery mode.
Thanks for the info on the boot order. I didn't realize the radio image was used for booting, I figured it was just firmware for the 3G chip or something.
I'm getting close to being able to apply my own update.zip style update. I've been able to reflash the recovery partition with a custom built recovery image that skips the signature verification. I'm having a touch of trouble getting it to actually install an update.zip though. It keeps saying "update script not found", even though there is a "META-INF/com/google/android/update-script" file in the zip. Arg! Anyways.. I'm in the process of tracking the issue down. More to come!
cmonex said:
that partition is the config partition, standard HTC stuff.
T-MOB010 is your CID (carrier ID, spl checks this when flashing NBH), but it is only a backup copy of it, the real CID is in radio part of nand (protected!).
and the rest is just config stuff for SPL and radio. (yes HTC uses strings to set these configs - SPL reads the strings from specific nand addresses and decides what to do)
Click to expand...
Click to collapse
Ah! Excellent info.
I was finally able to get the rebuilt recovery tool to work. The problem with the update.zip was due to the fact that the zip was built in windows, so it had the wrong path separators. It was looking for META-INF/com/google/android/update-script, but the zip file contained META-INF\com\google\android\update-script
After switching the slashes around in a hex editor, it installed the update no problem.
Next, I was tempted to try to flash the boot partition. I was somewhat sure that I would be able to recover from a bad boot flash, with my nifty new recovery tool. But after reading about the guy that bricked his phone (over in the root thread), I was a bit scared.
But I finally went ahead and decided to give it a try. You only live once, right?
So I opened up a root console, and just wiped the boot partition clean. "cat /dev/zero > /dev/mtd/mtd2" and rebooted. Palms sweaty.. breathing hard.. shaking.. Powered the phone on (without holding down home), and it comes up to the recovery tool. So far so good. Then I ran the update on the sdcard - it was a smallish update I had created before hand that just flashed the original boot image back to mtd2. Update runs fine.. phone reboots....
And it boots up normally.
yes!
*takes a big sigh of relief*
So now I can mess around with the boot partition, and know that I have that recovery tool safety net.
And now. It's time to sleep. *head hits the keyboard*
very cinematic good job mate!
Very nice work
Congrats on the successful flash!
strings in mda1.img include:
Code:
ANDROID!
no_console_suspend=1
-- System halted
ran out of input data
Malloc error
Memory error
Out of memory
incomplete literal tree
incomplete distance tree
bad gzip magic numbers
internal error, invalid method
Input is encrypted
Multi part input
Input has invalid flags
invalid compressed format (err=1)
invalid compressed format (err=2)
out of memory
invalid compressed format (other)
crc error
length error
Uncompressing Linux...
done, booting the kernel.
What are these images? Are they filesystem images that you could theoretically mount? If so, what filesystem (I haven't gotten anything to work).
Here is a zip file containing a modified recovery and boot image, as well as a few other things:
http://rapidshare.com/files/166164961/AndroidMod.zip
http://jf.nyquil.org/AndroidMod.zip
http://android-dls.com/forum/index.php?f=24&t=191&rb_v=viewtopic (see post for actual link to file)
NOTE: if you need a complete RC30 to v1.3 guide, see this page.
The recovery image (recovery_testkeys.img) uses the test keys that are distributed with the android platform source. This means that an OTA update or an update.zip update must be signed with the test key in order for it to install. In other words, it will no longer install OTA updates from t-mobile. You don't want them stealing back root access from you now do you? .
I've also included the test keys and the SignApk.jar tool, so you can sign your own update scripts (for use only with the modified recovery image). You can resign any image, even if it has been signed before. So for example, if you needed to install an "official" t-mobile update, you must re-sign it with the test keys first.
Another bonus in this recovery image is that ADB is enabled while in recovery mode. You can't adb into a shell (no sh binary), but you can at least use it to push and pull files from the device. For example, you could push an update.zip file to the sdcard.
The boot image (boot_nosecure.img) has been modified so that adb has root access by default. So when you do an adb shell, you automatically get a root shell. You can remount the system image using adb, and then push files directly to the system partition.
Finally, the "update - Restore Original RC29 Boot Image.zip" file is an update.zip file signed with the test keys, which will restore your boot partition back to the stock RC29 image. Useful if you accidentally hose your boot partition..
To install the recovery image onto your phone:
Code:
D:\Android\AndroidMod>adb push recovery_testkeys.img /data/local/recovery.img
912 KB/s (0 bytes in 1767424.001s)
D:\Android\AndroidMod>adb shell
$ su
su
# mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system
mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system
# cd /system
cd /system
# cat /data/local/recovery.img > recovery.img
cat /data/local/recovery.img > recovery.img
# flash_image recovery recovery.img
flash_image recovery recovery.img
#
Note: You must place the recovery image at /system/recovery.img. the init.rc boot script automatically flashes the recovery partition with that file every time you boot up the phone.
At this point, it's probably a good idea to reboot the phone into recovery mode, and make sure it loads OK. If the recovery image is corrupt somehow, it will throw you back into SPL mode (the multi-color bootloader screen). If that happens, just boot the phone normally, and reflash recovery image.
Once it boots into recovery mode, press alt+L, and the next to top line of text should say something like "using test keys.". If it doesn't, then you're still using the original recovery image.
Note: If you are planning on installing the modified RC30 update, you can ignore the following - there is no need to install the boot image. The update already has a newer, modified boot image.
Now that you know you have the modified recovery image loaded, you can install the boot image:
Code:
D:\Android\AndroidMod>adb push boot_nosecure.img /data/local/boot.img
939 KB/s (0 bytes in 1533952.001s)
D:\Android\AndroidMod>adb shell
$ su
su
# flash_image boot /data/local/boot.img
flash_image boot /data/local/boot.img
# rm /data/local/boot.img
rm /data/local/boot.img
#
Now reboot the phone and let it boot normally. If the boot image was corrupted, it will boot into recovery mode instead. You can use the included update zip file to reload the original RC29 boot image.
Otherwise, if it boots up normally, open a command prompt however you like (telnet, adb, terminal emulator app, etc.) and type "getprop ro.secure". If it says 0, then you're running the modified boot image. Otherwise, if it says 1, you're still running the original boot image.
Attachement..
Hmm. It doesn't look like the attachment made it.. Does anyone have some space I could throw the file up at? It's around 5mb.
JesusFreke said:
Hmm. It doesn't look like the attachment made it.. Does anyone have some space I could throw the file up at? It's around 5mb.
Click to expand...
Click to collapse
I should have some space let me know
JesusFreke said:
Hmm. It doesn't look like the attachment made it.. Does anyone have some space I could throw the file up at? It's around 5mb.
Click to expand...
Click to collapse
Sent you an email with u/p if you need space.
test
anyway to test and make sure i did this correctly.. other than my phone booted and is not a paperweight
jriley60 said:
anyway to test and make sure i did this correctly.. other than my phone booted and is not a paperweight
Click to expand...
Click to collapse
To check the boot image, boot the phone normally, and then get a shell with adb. Type "id", and see if you are root
To check the recovery image, boot up the phone into recovery mode. Once you're in recovery mode, Press alt-l to show the text. The next to top line should say something like "using test keys"
JesusFreke said:
To check the boot image, boot the phone normally, and then get a shell with adb. Type "id", and see if you are root
To check the recovery image, boot up the phone into recovery mode. Once you're in recovery mode, Press alt-l to show the text. The next to top line should say something like "using test keys"
Click to expand...
Click to collapse
When will we see the files? Can you just upload to RS and we will mirror?
neoobs said:
When will we see the files? Can you just upload to RS and we will mirror?
Click to expand...
Click to collapse
Look at the original post. I added a link for the zip file..
boot.img is in correct. assuming typing id in telnet returning uid=0(root) gid=0(root) means i'm root then i'm good, thank you so much. looks like i really should install the emulator it might make things a little easier
JesusFreke said:
Look at the original post. I added a link for the zip file..
Click to expand...
Click to collapse
thank you. Sorry
jriley60 said:
boot.img is in correct. assuming typing id in telnet returning uid=0(root) gid=0(root) means i'm root then i'm good, thank you so much. looks like i really should install the emulator it might make things a little easier
Click to expand...
Click to collapse
Well, that does mean you have root, but that doesn't say anything about whether the boot.img was installed correctly. If you're telneting in, then you would have root access regardless of whether you are running a stock boot image or my modified one.
My boot image allows adb to connect to the phone as root. If you don't use adb, there's no reason to install my modified boot image.
Actually, there's an easier way to tell if you're running my boot image. Get to a command prompt (telnet, adb, terminal emulator app, whatever), and type
getprop ro.secure
If it says 0, then you correctly installed my boot image. Otherwise, if it says 1, you're still running the stock image.
Thanks! Was waiting for this.
Now to screw with my phone like crazy
Not that I don't trust you... but...
Ok... I don't trust you implicitly enough to reflash my phone with your stuff
Any chance you can post diffs against the android source tree so I can apply your changes and build it myself?
No offense - I just like to know what's going on...
RyeBrye said:
Ok... I don't trust you implicitly enough to reflash my phone with your stuff
Any chance you can post diffs against the android source tree so I can apply your changes and build it myself?
No offense - I just like to know what's going on...
Click to expand...
Click to collapse
Not at all
The recovery tool is just a stock build (almost) from the android source, using the test keys, which is the default if you don't specify keys of your own. The only change I made was to make it print out "using test keys" when it runs, just to make it easy to tell if it's running. I can give you a diff if you really want.. but it's a simple change though, and doesn't affect the actual functionality.
For the boot image, I replaced the initramfs image in the boot.img included in the official RC29 update, with the initramfs image from a default build of the android source, which has the ro.secure property set to 0.
I first tried the boot.img that was generated by the default android build, but I had issues with getting wifi to work, so I tried merging the initramfs image with the RC29 boot.img, and it seems to work fine.
I suspect you could accomplish the same thing by extracting the initramfs image from the RC29 boot.img, un-gzipping and un-cpioing it, and then modifying the default.prop file to set ro.secure to 0. Then you would have to package it back up and stick it back into the RC29 boot.img.
ro.secure is the property that the adb service looks at to determine if it should use root user, or drop to the shell user. When ro.secure is 0, adb will run as root.
It can be a bit of a pain to get the android source to build though. Make sure you get the dream specific product files (they aren't downloaded by default when you do a "repo sync"). You'll also encounter issues where it can't find libaudio.so or librpc.so. You'll have to copy these from the phone to a couple of output folders in order for the build to proceed.
If you get stuck, feel free to give me a holler and I'll try and help out.
Be warned.. the build takes quite a while.. on the order of an hour or two at least. But then again, I was doing it in a VM.. it may be faster if you do it on a native linux box.
i cant get it to work i know I'm doing something wrong can you(everyone)help me out(i have Vista 64) i get this error
this i what i type# C:\Android\AndroidMod>adb push recovery_testkeys.img /data/local/recovery.img
this is the errorC:AndroidAndroidMod: not found
please and thank you
EDIT: could we do it off the sdcard?
EDIT2: i think i found my own mistake this cant be done in Windows i need to have shell with adb meaning time to whip out VM
JesusFreke said:
For the boot image, I replaced the initramfs image in the boot.img included in the official RC29 update, with the initramfs image from a default build of the android source, which has the ro.secure property set to 0.
I first tried the boot.img that was generated by the default android build, but I had issues with getting wifi to work, so I tried merging the initramfs image with the RC29 boot.img, and it seems to work fine.
Click to expand...
Click to collapse
Can you talk more about this step of the process? How did you do this "merging"? Did you use mkbootimg?
JesusFreke said:
I suspect you could accomplish the same thing by extracting the initramfs image from the RC29 boot.img, un-gzipping and un-cpioing it, and then modifying the default.prop file to set ro.secure to 0. Then you would have to package it back up and stick it back into the RC29 boot.img.
Click to expand...
Click to collapse
And this could be done without going through the whole process of doing an Android build, right? I'm just thinking about how one might build a simple utility to allow editing of the ramdisk.
alansj said:
Can you talk more about this step of the process? How did you do this "merging"? Did you use mkbootimg?
Click to expand...
Click to collapse
I just used the good ol hex-editor. The gzip file starts with a few specific bytes (don't remember them offhand..), so you can search through the image. There are 2 gzip files, the initramfs is the last one. In mine, it starts at offset 0x00154000.
Once you find it, just cut it out and dump the new one in (there is some 00 padding after the gzip file ends.. not sure if you need to keep the padding or not). You also have to update the size of the initramfs, which is at offset 0x00000010.
alansj said:
And this could be done without going through the whole process of doing an Android build, right? I'm just thinking about how one might build a simple utility to allow editing of the ramdisk.
Click to expand...
Click to collapse
Yes.
Anyway to make this using the update.zip sd card method?
JesusFreke, per some requests in #android on freenode I have setup a wiki (not a device wiki like xda's but more like an "information about android/g1 and how to tweak it" wiki) and would like to put this on there. Let me know if you care (unless you would like to add it in your own words), the wiki is http://android-dls.com/wiki and its still very new, but im trying to to get it built up (RyeBrye is doing most of the work).
humble said:
i cant get it to work i know I'm doing something wrong can you(everyone)help me out(i have Vista 64) i get this error
this i what i type# C:\Android\AndroidMod>adb push recovery_testkeys.img /data/local/recovery.img
this is the errorC:AndroidAndroidMod: not found
please and thank you
EDIT: could we do it off the sdcard?
Click to expand...
Click to collapse
First of, you do know that when we refer to "C:\..." we refer to windows via a command prompt (or "cmd") and when you see "# ..." we refer to a shell connection to the Android phone.
Second, you do have the Android SDK right? If not download it HERE. Now extract that to a folder, preferably close to the C: root. (ex. C:\AndroidSDK)
1)Either extract/copy the files from the "AndroidMod.zip" to the Android Tools folder from the SDK (ex. C:\AndroidSDK\Tools) OR copy "adb.exe" and "AdbWinApi.dll" from the Android Tools folder from the SDK (ex. C:\AndroidSDK\Tools)to the folder where you have extracted the "AndroidMod.zip" to.
2)Open up a command prompt. Start -> Run (or Windows key + R) and type "cmd"
3)CD to the directory where the files are.
EXAMPLE:
C:\Users\[your_user_name_here]> cd \
C:> cd androidsdk\tools
C:\AndroidSDK\Tools>
4) Now follow the Instructions.
Several people have already figured out the details on their own, but I have gotten requests to do a more comprehensive tutorial on how the boot and recovery images are structured, and how you can edit them.
Background
Your phone has several devices which hold different parts of the filesystem:
Code:
#cat /proc/mtd
dev: size erasesize name
mtd0: 00040000 00020000 "misc"
mtd1: 00500000 00020000 "recovery"
mtd2: 00280000 00020000 "boot"
mtd3: 04380000 00020000 "system"
mtd4: 04380000 00020000 "cache"
mtd5: 04ac0000 00020000 "userdata"
In this tutorial, we will deal with "recovery" and "boot". The "boot" device holds the files that are automatically loaded onto the root of your filesystem every time you boot (details below).
"system" holds everything that gets mounted in your system/ directory, and userdata/ is everything that shows up in data/ (this is all the apps you've installed, your preferences, etc).
The recovery and boot partitions are at /dev/mtd/mtd1 and /dev/mtd/mtd2, and before you do anything else you should back these up (note: this may not be the best way of doing this because it may not deal properly with bad blocks etc, but it's all we've got until somebody comes up with a better method, and besides you will probably be restoring from update.zip anyway):
Code:
# cat /dev/mtd/mtd1 > /sdcard/mtd1.img
# cat /dev/mtd/mtd2 > /sdcard/mtd2.img
The other thing you should do is put your favorite update.zip file into the root directory of your sd card so that if you screw up your boot partition you can boot into recovery mode and re-apply the update. You probably want one of the pre-rooted recovery images found elsewhere on the forums.
There is also another important file you should know about. In /system/recovery.img there is a full copy of everything that is loaded on mtd1. This file is automatically flashed onto mtd1 every time you shut down. That means two things: 1. Any changes you make directly to /dev/mtd/mtd1 get blown away on reboot and 2. If you want to change /dev/mtd/mtd1 you're probably better off just sticking the image in /system/recovery.img and rebooting. When creating your own custom update.zip files (especially when adapting the stock images), you can get tripped up if you forget to replace /system/recovery.img and it ends up overwriting /dev/mtd/mtd1 unbeknownst to you. Watch out.
Structure of boot and recovery images
The boot and recovery images are not proper filesystems. Instead, they are a custom android format consisting of a 2k header, followed by a gzipped kernel, followed by a ramdisk, followed by a second stage loader (optional, we have not seen these in the wild yet). This structure is outlined in mkbootimg.h:
Code:
+-----------------+
| boot header | 1 page
+-----------------+
| kernel | n pages
+-----------------+
| ramdisk | m pages
+-----------------+
| second stage | o pages
+-----------------+
n = (kernel_size + page_size - 1) / page_size
m = (ramdisk_size + page_size - 1) / page_size
o = (second_size + page_size - 1) / page_size
0. all entities are page_size aligned in flash
1. kernel and ramdisk are required (size != 0)
2. second is optional (second_size == 0 -> no second)
A ramdisk is basically a small filesystem containing the core files needed to initialize the system. It includes the critical init process, as well as init.rc, which is where you can set many system-wide properties. If you really want to know more about it, here is the documentation. Here's a list of files on a typical ramdisk:
Code:
./init.trout.rc
./default.prop
./proc
./dev
./init.rc
./init
./sys
./init.goldfish.rc
./sbin
./sbin/adbd
./system
./data
The recovery image typically has a few extra files, which constitute the recovery binary and supporting files (the application that gets run if you hold down home+power when rebooting). These files are:
Code:
./res
./res/images
./res/images/progress_bar_empty_left_round.bmp
./res/images/icon_firmware_install.bmp
./res/images/indeterminate3.bmp
./res/images/progress_bar_fill.bmp
./res/images/progress_bar_left_round.bmp
./res/images/icon_error.bmp
./res/images/indeterminate1.bmp
./res/images/progress_bar_empty_right_round.bmp
./res/images/icon_firmware_error.bmp
./res/images/progress_bar_right_round.bmp
./res/images/indeterminate4.bmp
./res/images/indeterminate5.bmp
./res/images/indeterminate6.bmp
./res/images/progress_bar_empty.bmp
./res/images/indeterminate2.bmp
./res/images/icon_unpacking.bmp
./res/images/icon_installing.bmp
./sbin/recovery
Unpacking, Editing, and Re-Packing the images
Note: below I give you the details for unpacking and repacking manually, but I have attached two perl scripts that do most of this for you
If you are good with a hex editor, you can open up any of these images and strip off the first 2k of data. Then, look for a bunch of zeroes followed by the hex 1F 8B (which is the magic number of a gzip file). Copy everything from the first line of the file, through the zeroes, and stopping at the 1F 8B. That is the kernel. Everything from the 1F 8B through the end is the ramdisk. You could save each of these files separately. In order to see the contents of the ramdisk, you need to un-gzip it and then un-cpio it. You could use a command like this (ideally after creating a new directory and cd'ing into it):
Code:
gunzip -c ../your-ramdisk-file | cpio -i
That will place all of the files from the ramdisk in your working directory. You can now edit them.
In order to re-create the ramdisk, you need to re-cpio them and re-gzip those files, with a command like the following (remember, cpio will include everything in the current working directory, so you probably want to remove any other cruft you might have in there):
Code:
find . | cpio -o -H newc | gzip > ../newramdisk.cpio.gz
The final step is to combine the kernel and your new ramdisk into the full image, using the mkbootimg program (which you should download and compile from the git repository):
Code:
mkbootimg --cmdline 'no_console_suspend=1 console=null' --kernel your-kernel-file --ramdisk newramdisk.cpio.gz -o mynewimage.img
Now, there's a lot of hassle in pulling apart files in hex editors and remembering all of these commands, so I wrote unpack and repack perl scripts for you (attached). Hooray.
Flashing your new image back onto the phone
You will probably only ever be flashing boot images directly to the phone, given the fact that /system/recovery.img automatically flashes the recovery device for you (as noted above). If you have created a new recovery image, just stick it in /system/recovery.img and reboot. If you are flashing a boot image, stick it on your phone via adb (a tool included in the Android SDK):
Code:
adb push ./mynewimage.img /sdcard
Then, open a shell to your phone via 'adb shell', get root, and do the following two commands to flash your new boot image:
Code:
# cat /dev/zero >> /dev/mtd/mtd2
write: No space left on device [this is ok, you can ignore]
# flash_image boot /sdcard/mynewimage.img
Reboot.
If your phone starts all the way up, congratulations. If not, you did something wrong and you'll need to boot into recovery mode and apply your update.zip file (reboot while holding down home+power, when you get the recovery screen press alt+L and then alt+S).
Something fun to do with your new found power
If you place a file titled initlogo.rle in the root directory of your boot image, the phone will display this image upon boot (after the "G1" image and before the Android animation). In order to create this file, you need to create a 320x480 image in Photoshop or Gimp and save it as a "raw image" file. You then need to compress that image with the program to565. More details on that here.
This is not the same thing as applying an update.zip
You will see other places on the forums that describe how to create customized update.zip files, as well as update.zip files that people are sharing. For example, there is a recent update.zip which is a modified version of rc30 (with the anti-root aspects disabled). The update.zip files include new boot images, recovery images, and typically replacements for the entire system/ directory as well as other updates. If you are creating a custom boot or recovery image, it is typically a good idea to start with the image distributed with the most recent update you have applied (flashing an image from an older release could have unintended consequences).
Questions?
hooray! you're awesome
Where does boot.img flash? What is the corresponding part of the system?
Dimath said:
Where does boot.img flash? What is the corresponding part of the system?
Click to expand...
Click to collapse
I'm not sure what exactly you mean, but when you do flash_image boot imagefile.img it will write imagefile.img to /dev/mtd/mtd2, which is where your phone looks for the boot files. Did that answer your question?
For your command...
Code:
cat /dev/zero >> /dev/mtd/mtd2
Do you mean
Code:
cat /dev/zero > /dev/mtd/mtd2
?
The idea being that you erase flash in the version with one '>', whereas you... append to the end of a device in the version with two '>'s? I can see the utility of erasing flash with one '>' but appending seems... odd. Am I missing something?
Is this any different than using the Dalvik Debug Monitor (DDMS) file manager found the the Android SDK? I'm able to push, pull, and delete files on my G1 with no problem.
Would this be the only way to rebuild a system app (i.e. Settings.apk) with more debug (Log. to extract via adb logcat over usb), then rebuild the entire system.img, then flash into the G1?
eckzow said:
For your command...
Code:
cat /dev/zero >> /dev/mtd/mtd2
Do you mean
Code:
cat /dev/zero > /dev/mtd/mtd2
?
The idea being that you erase flash in the version with one '>', whereas you... append to the end of a device in the version with two '>'s? I can see the utility of erasing flash with one '>' but appending seems... odd. Am I missing something?
Click to expand...
Click to collapse
I think you're right. I copied that from somebody else's instructions and it certainly seems to make more sense with one '>'. Anybody know for sure?
In any event, this is unnecessary in most cases because flash_image should overwrite the whole thing. The only exception is when you have an identical header on your image to the one that is already on the device. This shouldn't happen in my instructions (mkbootimg creates a header that includes the build timestamp) but I kept the instruction there just for good measure.
andonnguyen said:
Is this any different than using the Dalvik Debug Monitor (DDMS) file manager found the the Android SDK? I'm able to push, pull, and delete files on my G1 with no problem.
Click to expand...
Click to collapse
The adb push command I gave is no different, but you would still have to unpack/repack and flash_image according to my instructions.
I've tried these instructions on RC30 1.3. Basically I extracted, unpacked, and repacked, just to see if it would work. The resultant file is too large; it fails when you run flash_image.
I'm trying to modify my boot image of my ADP1 using the perl scripts, but I receive the following warning while decompressing the ramdisk:
Code:
$ unpack-bootimg.pl mtd2.img
kernel written to mtd2.img-kernel.gz
ramdisk written to mtd2.img-ramdisk.cpio.gz
removed old directory mtd2.img-ramdisk
[B]gzip: ../mtd2.img-ramdisk.cpio.gz: decompression OK, trailing garbage ignored
462 blocks[/B]
extracted ramdisk contents to directory mtd2.img-ramdisk/
Is this warning expected? Is safe to continue?
Also, I've found that the size of the modified packed image is far smaller than the original one
Code:
$ ll mtd2.img mtd2-modified.img
-rwx------ 1 ris ris 2621440 2009-01-07 19:19 mtd2.img
-rw-r--r-- 1 ris ris 1533952 2009-01-07 20:49 mtd2-modified.img
Update: I've tried anyway.
For the record, I've obtained the boot.img using cat, uncompressed with the perl script, modified default.prop, repacked with the perl script.
As you see from the code above, the img file is much smaller (Opening with a hex editor you can see that the end of the original image is full of 0xFF, so I believe it's ok, both the gzip warning and the different file sizes).
Reflashed it from recovery mode, using
fastboot flash boot mt2-modified.img
fastboot reboot
... and worked flawlessly
I'm leaving the coment for future references.
Thanks for the tutorial
[RiS] said:
gzip: ../mtd2.img-ramdisk.cpio.gz: decompression OK, trailing garbage ignored
462 blocks
Is this warning expected? Is safe to continue?
Click to expand...
Click to collapse
Yes, you would expect trailing zeroes, which would give you that error. The trailing zeroes exist in order to pad the image size to the nearest page boundary. They are added by mkbootimg.
[RiS] said:
Also, I've found that the size of the modified packed image is far smaller than the original one
As you see from the code above, the img file is much smaller (Opening with a hex editor you can see that the end of the original image is full of 0xFF, so I believe it's ok, both the gzip warning and the different file sizes).
Click to expand...
Click to collapse
Yep, that's exactly why. Nothing to worry about.
Thanks for the informative clarification.
alansj said:
There is also another important file you should know about. In /system/recovery.img there is a full copy of everything that is loaded on mtd1. This file is automatically flashed onto mtd1 every time you shut down. That means two things: 1. Any changes you make directly to /dev/mtd/mtd1 get blown away on reboot and 2. If you want to change /dev/mtd/mtd1 you're probably better off just sticking the image in /system/recovery.img and rebooting.
Click to expand...
Click to collapse
I'm using an stock ADP1, and the file /system/recovery.img does not exist. Is this expected?
Also, I've found out in JFv.1.31 that the recovery image is in /data/recovery.img (although there is no /data/recovery.img in my ADP1 neither..)
[RiS] said:
I'm using an stock ADP1, and the file /system/recovery.img does not exist. Is this expected?
Also, I've found out in JFv.1.31 that the recovery image is in /data/recovery.img (although there is no /data/recovery.img in my ADP1 neither..)
Click to expand...
Click to collapse
It deletes it after it flashes on the first bootup after you apply the update.
JesusFreke said:
It deletes it after it flashes on the first bootup after you apply the update.
Click to expand...
Click to collapse
Are you refering to /data/recovery.img or /system/recovery.img? or both?
[RiS] said:
Are you refering to /data/recovery.img or /system/recovery.img? or both?
Click to expand...
Click to collapse
I'm referring to /data/recovery.img in JFv1.2 and up (although there was a bug in the RC8 version of JFv1.2 that prevented it from being deleted)
So is there any reason why there is no /system/recovery.img on my ADP1?
When I use the unpack script on JF1.31 boot.img, it prints out like this:
"Could not find any embedded ramdisk images. Are you sure this is a full boot image?"
Any help?
[RiS] said:
So is there any reason why there is no /system/recovery.img on my ADP1?
Click to expand...
Click to collapse
Stock or JFv1.3?
JesusFreke said:
Stock or JFv1.3?
Click to expand...
Click to collapse
"Stock". I've just modified boot img to change default.prop
I did not make this.
When messing with mytouch3G the old way to root required this .img.
But I put it on my G1.
I put it on my phone by flashing through adb under tools
too try it out
cd\
cd androidsdk\tools\ *or whatever you named your sdk stuff
fastboot boot recovery-RAv1.2.1G.img
to put it on your g1
cd\
cd androidsdk\tools\ *or whatever you named your sdk stuff
fastboot erase recovery
fastboot flash recovery recovery-RAv1.2.1G.img
takeout battery put it back in boot into recovery and enjoy.
*new features
-format sd fat32/ext2/swap
-format sd fat32
-convert ext2 to ext3
-fix package uid mismatches
-move apps+dalvik-cache to sd
same old features *cy's recovery features* no terminal tho!
-reboot system
-apply sdcard update.zip
-apple sdcard choose zip
-wipe data/factory reset
-nandroid v2.2 backup
-nandroid v2.2 restore
-repair ext filesystem
Download here!
http://www.megaupload.com/?d=0MW0CIU2
sticking with cyanogen until you can list the features better...plus does this contain the features of cyan 1.4 in addition to those things u listed?
Adidas Reborn said:
I did not make this.
When messing with mytouch3G the old way to root required this .img.
But I put it on my G1.
I put it on my phone by flashing through adb under tools
start
cmd *enter
cd\
cd androidsdk\tools\ *or whatever you named your sdk stuff
fastboot erase recovery
fastboot flash recovery recovery-RAv1.2.1G.img
takeout battery put it back in boot into recovery and enjoy.
*new features
-format sd fat32/ext2/swap
-format sd fat32
-convert ext2 to ext3
-fix package uid mismatches
-move apps+dalvik-cache to sd
Download here!
http://www.megaupload.com/?d=0MW0CIU2
Click to expand...
Click to collapse
what features does it include?
yah its a build off cy's just more stuff like an easy way to format your sd without so much work to do it and thats a big key
confirmed more features! but no option to boot into terminal!
thats a no no! back to cy 1.4
well its new.. maybe someone can work off this. to add the terminal.
after all it was based for the mt3g but works with g1
Indeed new features
Agreed with drizzy i will switch back to cyan but for those of you who will ask in a matter of time you can flash this from your terminal by
1. put image into root of sdcard
2. open terminal and type in the following
a. mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system (press enter)
b. cd sdcard (press enter)
c. flash_image recovery recovery-RAv1.2.1G.img (press enter)
would be great recovery if had console.
It's a real nice recovery based off of cyanogen's 1.4 so maybe he can add console and make 1.5 for us real easily
yeah i like how it includes the farmat of sd automatically which alot of people like just need terminal so yeah
This is where it came from!
http://forum.xda-developers.com/showthread.php?t=530492
xda-developers > Sapphire > Sapphire android development > [Updated 8/23] [Modded recovery] RAv1.2.1
Amon_RA said:
Make sure your phone is rooted and that you are using an engineerings SPL!
Features
root
full ADB access in recovery mode
Partition your SD directly from the menu! (500MB Ext2, 32MB Swap, remaining Fat32)
Repair ext filesystem, convert ext2 to ext3, fix package mismatches, move apps2SD directly from the menu!
Flash ANY .zip from your SD - by krut0n
51dusty's sdparted v0.5 util *NEW
Loads of scrips and binaries from Cyanogen's v1.4 dream recovery
Nandroid v2.2 backup and restore
Busybox
Green colored menu with a nice HTC logo
See detailed changelog for more details!
Download
Use the H/G version if your Magic is HTC branded (PVT 32A board)/Google branded (PVT 32B board, or if you have an HTC Dream).
Latest :
RAv1.2.1H - MD5Sum: F5858017CE32B6E959313B6C970AC6D3
RAv1.2.1G - MD5Sum: 965497CC010CAA0326A4B26D2F25CBED
Old versions :
RAv1.2.0H - MD5Sum: D819FABD43EBC1A0B6FA4AABFADD3116
RAv1.2.0G - MD5Sum: F5919CB4E2D406C11BDF7707A90423BF
RAv1.1.2H - MD5Sum: 2C7B9771B3598EE4007F06BCB4B1DA61
RAv1.1.2G - MD5Sum: 16DFA87E7B583FDD9E151E857A8C5ECE
How to use/install
Try it out WITHOUT making changes to your device (NO flashing):
Code:
fastboot boot recovery-RAv1.2.1H.img
Flash it permanently (try it first by following the steps above and making a nandroid backup first!):
Code:
adb remount
adb shell rm /system/recovery.img
OR (depending on where your ROM is storing the recovery image)
adb shell rm /data/recovery.img
Code:
fastboot erase recovery
fastboot flash recovery recovery-RAv1.2.1H.img
Special thanks to :
51dusty for his super sdparted script! Thanks! ^^
Cyanogen for his dedicated work! , buy him a beer!
Haykuro for finding the HTC offsets ^^ , buy him a Nesquik!
JesusFreke for his awesome JFv1.43's recovery , donate
Click to expand...
Click to collapse
did anyone try it to see if it works, and if its worth upgrading?
since there is a way to utilize this without flashing it to your phone it would be a great way to quickly partition an sdcard. especially for all the noobs over at the Q&A thread who for some reason cannot type a few commands into the recovery console
if it had an option to format just ext3 and swap id be down...but I dont wanna have to clear out my fat32 all the time.
I created this recovery specifically for the Sapphire => no keyboard!
The fact that the Sapphire doesn't have a keyboard is the only reason why I didn't include any keyboard related stuff like the terminal...
My original recovery was created so people could root their 32A Sapphire (the code was based on the AOSP cupcake release).
After Cyanogen's improvements I decided to build further on his recovery source code and make it more "Sapphire friendly", hence the one-click options.
I'm not really planning to release a specific recovery for the Dream (including keyboard options + terminal) because my 32B version will boot on the Dream anyway...
Amon, just want to thank you for your work ever though it was not meant for us g1 users it would be a great help to all the noobs to have it, just for the ease of it for them
hm add a feature to wipe ext3, boot into parted, and add terminal and this is a keeper.
Seems that a refurbished G1 I received has 3 blocks stuck and marked in eccUnfixed for some time, pulled the info from command:
Code:
cat /proc/yaffs
Issues:
I'm thinking as our G1s are getting older and with ROM flashing abuse we are killing NAND blocks on them fast. Researching at various sites linux-mtd & Yaffs.net, the garbage collector of Yaffs should be marking and removing these if Yaffs was controlling the ECC mode. However since Yaffs G1 android setting relies on NANDecc for marking bad blocks I'm not so sure which is cleaning up and not retiring these blocks, maybe NAND and YAFFS are in disagreements or one has ability to retire these blocks and end this problem. It seems though that while flashing ROMs data is being written on to these areas and after a remount it is evident that something may be wrong.
Experiences:
An app could go missing or a file or folder name with invalid characters may happen or a phones feature may suddenly stop workin such as scanning for wireless networks timing out.
The issue is reproducable on freshly erased partitions via fasboot erase system -w.
If a dev would like to co-ordinate with me I will grant them remote access to a system with said phone connected via usb with adb port redirection/remote adb or desktop access with adb and fastboot and vnc client with vnc server on phone .
Workaround:
Working with xda member Licknuts we tested numerous times with a few ROMs 1.6 and 2.1 and came up with this for a quick work around that basically involves erasing /system via fastboot then in recovery lock /system from being formatted and unmounted after ROM Flashing you then tarball /system partition to /sdcard do a reboot into fastboot and erase system again reboot into recovery and untarball back to /system.
Prerequisite:
A phone that suffers missing apps in /system or odd phone behavior features after fresh ROM flash.
ADB with Fastboot, USB Cable connected to your Phone.
Adapt the procedure bellow for issues stemming from the /data partition.
<Go for it>
Going to have to clear out system as it probably has corruption.
Code:
fasboot erase system
Note:::
Hboot spl dictates reboot & some engineer spls have no menu option and engineer spls do not reboot to recovery with an empty partitions when USB cable is connected so disconnect it and reboot then reconnect when in recovery console or just fastboot flash system system.img using HTC 1.6 system.img extracted from their zip.
FYI:::
Haykura sapphire spls (aka: danger spl / death spl) are able to boot to recovery with only radio/hboot/recovery partitions intact as long as it is instructed to do so other wise you get G1 screen with red[fastboot] with USB cable attached (might be the missing safety mode for bricked G1s but not certain).
Anyhow just Reboot to recovery console from menu option or [send][menu][hang] quickly hold [home]
Using adb shell we can lock system partition from being formatted and unmounted by ROM Flash scripts instructions in scripts.
Code:
adb shell
mount -a
cd /system
flash ROM and any updates from Recovery console on phone as normal...... and then we tarball the /system partition before it remounts on reboot and corruption happens
While still in adb shell and hanging out in /system, issue these commands to mount the sdcard and create a tarball of /system.
Code:
mount /sdcard
tar -cvf /sdcard/system.tar /system
sync
reboot bootloader
in fastboot we erase system again because most likely data will corrupt because the Flashing procedure may have written to bad blocks and when mounted or remounted it will corrupt, this is why we tarballed /system before it happens.
Code:
fastboot erase system
then reboot back to recovery console
using adb shell we un-tarball system.tar back to /system mountpoint with YAFFS that supposedly knows about these bad blocks so writing filles out these should be checked and avoided.
Issue these commands:
Code:
mount /sdcard
mount /system
tar -xvf /sdcard/system.tar -C /
sync
reboot
reboot as normally would and with some luck files wrote out avoiding those marked bad blocks.
Resolutions and notes:
flash_image and ROM flashing scripts may be flashing directly to device level which does not have YAFFS bad block information and once mounted with a file system, corruption happens from file system having bad block information.
Using sapphire spls (danger spl) we have access to fasboot oem task 28 & fastboot oem task 29 will low level format, repair, report and mark blocks bad but doesn't seem to resolve issues maybe YAFFS doesn't sync or care about the blocks marked in NAND mtd.
Possibly we need a compiled kernel and complete compiled tool set to fully work on Nand device mtd and format tools for Yaffs and scripts to automate some tasks. Seems Nandroid and /system/xbin, /sbin, /system/bin and recovery console tool-sets are limited as well.
*I think and type fast so note that typos may exists hopefully not in coding but elsewhere and if you exist solely to point out grammar and spelling of others on internet forums or broken English of members from other countries, then respect my preemptive "go **** yourself".
If my help aided in getting your phone in working order then give a family member or friend a call or donate some $ to the devs that keep our phones fun to use.
Have you got any more in-depth info on this?
The way I see it, you erase with fastboot... What does this accomplish that doesnt happen if one from the recovery console or adb, dd /dev/zero into the MTD device?
Or, better yet, wouldn't tarring the system partition after its written just copy the bad files that existed on the bad erase block(s)?
Or am i seriously not understanding what your instructions are doing.. To me:
Erase in some way once. What this does, i'm unsure.
Flash
Mount /system and tar to sdcard
Erase again (again what this does to assist the process is uncertain to me)
Boot to recovery
Mount everything
Untar from sdcard.
Tell me if I understand this wrong?
as some blocks are reserved to replace bad blocks , most of the time you won't see a problem
However, once these are used up you will effectively see partition 'shrinkage'
we can limit this 'shrinkage by using lbcoder's custom MTD via the kernel command line
I have a relatively simple way of implementing this
http://forum.xda-developers.com/showthread.php?p=7061471#post7061471
Thanks Phonekenstein for such a great input. I never knew this thread was exist until now but I found this and read the title, damn this is exactly what my case is!
Follow your instruction and re-install the rom, every single headached weird behaviors are disappear and the system runs like butter. It seems my phone is finally fixed!
I think this thread should get more attention.
(bump)
This is a great find! I will be trying this on my HD2 as I have some bad blocks also causing Android builds to go wack.