Automated NVFlash backup for Linux - G Tablet General

Inspired by http://forum.xda-developers.com/showthread.php?t=1010659
I set out to make a similar script for Linux. You still need nvflash installed and working. Nothing is changed on the tablet, but your mileage may vary, past performance is not an indicator of future results, and void where prohibited by law
Code:
#!/bin/bash
### Version 1.0 2 April 2011, wd5gnr
### Usage: dobackup
### Puts backup files $ROOT/backup/YYYYMMDDHHMM/
### by default
### Edit the variables below to set things up for your setup
###
### If you already have the bootloader running
### You can use dobackup -r
### To reconnect
# note: for some reason partition 6 and 11 on my device
# fail to read properly. 6 is the MSC partition on mine
# if I reformat it and reload it from stock, its ok. But
# something during running is blowing it again
# Partition 11 apparently is supposed to fail
# based on comments in the script
# from: http://forum.xda-developers.com/showthread.php?t=1010659
# because of that you may want to try the full PARTLIST
# and then edit to suit your situation
#PARTLIST=2 3 4 5 6 7 8 9 10 11
PARTLIST="2 3 4 5 7 8 9 10 11"
# Set the path for backups etc.
ROOT=~/android/nvflash
# Set the path for NVFLASH
NVFLASH=$ROOT/nvflash
# and bootloader
NVBOOT=$ROOT/bootloader.bin
# This gets the date in YYYYMMDDHHMM format
BSTAMP=`date +%Y%m%d%H%M`
# This picks the name of the backup (using the date)
BDIR=$ROOT/backup/$BSTAMP
# make directories
mkdir -p $BDIR
# I did not check to see that this succeeded
# we assume you have permission etc.
# if -r option we never have to load the bootloader
RESUME="--bl $NVBOOT"
if [ "$1" == '-r' ]
then RESUME=-r
fi
function donvf {
if $NVFLASH [email protected]
then true
else
echo NVFlash error 1>&2
rmdir $BDIR >/dev/null 2>&1 # delete an empty backup directory
exit 1
fi
}
# Get BCT first and prime the boot loader
donvf $RESUME --getbct --bct $BDIR/backup.bct --go
# get partition table
donvf -r --getpartitiontable $BDIR/parttable.txt
# for some reason #1 is not required or absent and #12 is the cache
# but you could add it if necessary (look at parttable.txt)
# Part 11 may give an error, but is ok
# If 11 gives error APX will be down and you'll have to reboot to
# do any more things
for I in $PARTLIST
do
if [ $I == 11 ]
then
echo You may safely ignore errors on partition 11 apparently 1>&2
fi
donvf -r --read $I $BDIR/part$I.img
done
exit 0 # because of partition 11, you won't get here unless you aren't doing 11
###
### Note BCT is hardware info (4080 bytes)
### Partition guesses
# 2 = BCT = ?
# 3 = PT = ?
# 4 = EBT = ?
# 5 = MBT = ?
# 6 = MSC = ? Some kind of recovery thing; 7 in stock
# 7 = BLO = Boot logo (6 in stock)
# 8 = OGO
# 9 = SOS = Recovery image (clockwork or stock)
# 10 = LNX = Linux?
# 11 = APP = Applications?
# 12 = CAC = Cache
Funny things are partition 6 (my MSC partition). I can reformat it and flash it with stock and it works. But later it will hang nvflash trying to read it. I can't find any info on what is in there anyway.
Also partition 11 gives an error, but apparently its not just me. See comments in the script.

Can you try running dmesg (e.g. via adb shell or using a terminal on the gtab), right after booting, then looking at the outout for bad block msgs?
I've been guessing that the reason is because of bad blocks...
Jim

MSC partition is actually the /misc folder, you can verify this by comparing content with the file inanop.log
inandop.sh create it, it is designed to make sure the partition is splited in 2 section 2/16 GB and run in init.rc
could be the reason people trying to partition 4GB/12 where having problem...
@ Jim, Seem like you are right, bad sector are really the source of read error in my unit...
I am not sure if we will encounter issue when writing from the OS since this is handled byt the actual file system but writing directly from nvflash seem to be bypassing any kinf of error/wear levelling provided by yaffs file system or even fat32
I would think that writing with the -v (Verify on) would be better to prevent write error but that's a feeling not based on fact (you need to disable this flag to restore a partition with bad block)
I also believe that bad block at the end of the partition 11 are not a concern, they are probably normal for this kind of inand memory... mine appear at 209.584Mb out of 209.715MB so the partition is still 209MB

Nothing in /misc looks exotic -- just a log file and a lost+found. Very strange that it hangs the nvflash read. I noticed it the other day and reformatted it from nvflash and rolled back the stock image and then I was able to read it back. But today it hangs my backup script again.
Oh yeah. And my dmesg log shows Falled Ddk Rd. Bad block error code=0xb0010 at chip=0 block=65. That repeats a few times. Not unique to me however: http://forum.xda-developers.com/showthread.php?t=933128

wd5gnr said:
Nothing in /misc looks exotic -- just a log file and a lost+found. Very strange that it hangs the nvflash read. I noticed it the other day and reformatted it from nvflash and rolled back the stock image and then I was able to read it back. But today it hangs my backup script again.
Oh yeah. And my dmesg log shows Falled Ddk Rd. Bad block error code=0xb0010 at chip=0 block=65. That repeats a few times. Not unique to me however: http://forum.xda-developers.com/showthread.php?t=933128
Click to expand...
Click to collapse
Hi,
Thanks for that info. If you look thru the dmesg, there'll be a listing of the partitions that the kernel is "creating" (I think, more like "assigning"), and I think that by looking at that, relative to the bad block msgs that will be interspersed, you can tell which partition (e.g., on my gtab, one of them is in /system) the bad block is occurring in.
Jim

P00r said:
MSC partition is actually the /misc folder, you can verify this by comparing content with the file inanop.log
inandop.sh create it, it is designed to make sure the partition is splited in 2 section 2/16 GB and run in init.rc
could be the reason people trying to partition 4GB/12 where having problem...
@ Jim, Seem like you are right, bad sector are really the source of read error in my unit...
I am not sure if we will encounter issue when writing from the OS since this is handled byt the actual file system but writing directly from nvflash seem to be bypassing any kinf of error/wear levelling provided by yaffs file system or even fat32
I would think that writing with the -v (Verify on) would be better to prevent write error but that's a feeling not based on fact (you need to disable this flag to restore a partition with bad block)
I also believe that bad block at the end of the partition 11 are not a concern, they are probably normal for this kind of inand memory... mine appear at 209.584Mb out of 209.715MB so the partition is still 209MB
Click to expand...
Click to collapse
Hi,
A couple of comments:
- The problem that I think may occur if you use the -v is that it may prevent the download of the partition to be completed. In other words, if you do --download with -v, and if nvflash hits a verify error in the middle of the partition, it may (I don't know) just stop at that point, then you end up with a half-populated partition, which could be pretty useless.
- Thanks for the info re. part 11, which I believe is "CAC", or "cache". I wonderinf if\ that partition really needs to be restored for a complete system restore?
BTW, what got me started with all of this was that way-back-when, when I first started working with my gtab, I wanted to find a way to do what I call a "complete bare-metal" backup/image of my gtab. That got me started working with nvflash, but then I ran into the --read problems.
As mentioned, one of the bad blocks on my gtab is in cache partition, which may or not needed for a full restore, but the worse thing is that the 2nd bad block that I have is apparently near the end of the system (/system) partition, so that prevents me from doing a successful read of the system partition to back it up.
I've tried a number of things, including doing repartitioning system using CWM, and also some messing with the system partition via adb while in CWM, i.e., I deleted the system partition completely, and the restored it. From what I've read, yaffs partitions don't need to be pre-formatted (don't need to be formatted before being used), so I was hoping (had seem some posts in some other forums) that that might eliminate the bad blocks if they were "soft", but it didn't.
So, I'm still stuck with 2 bad blocks on my gtab, and can't complete a full nvflash image of all the partitions on it ...
BTW, I just got a new gtab via woot, and I ran dmesg on it, and that too has 1 bad block, so it seems like probably all devices have some... Just some info...
Jim

wd5gnr,
So, I guess, at this point, is who is brave enough to try to do a full restore from the partition files that they pulled from their gtab (albeit with read errors)?
Have you tried that yet?
Just to be clear, I have NOT, mainly because of the read errors that I encountered when trying to backup, i.e., if I can't read the partitions, then I have doubts about the integrity of the files that I ended up with.
Jim

I have not. Partition 11 is the system partition, not the cache (at least on mine). On mine 12 is the cache.
If you look at the "inspired by" link above they mention that the image is still usable which implies there is slack space at the end maybe? But I don't know why/how they know that. And no, I have not reflashed my whole system from these backups although I have done a few on partition basis.

wd5gnr said:
I have not. Partition 11 is the system partition, not the cache (at least on mine). On mine 12 is the cache.
If you look at the "inspired by" link above they mention that the image is still usable which implies there is slack space at the end maybe? But I don't know why/how they know that. And no, I have not reflashed my whole system from these backups although I have done a few on partition basis.
Click to expand...
Click to collapse
I was in that thread you linked, and re-read it, and maybe I missed something, but I don't see where it says the image is still usable?
Jim

You have to open up the batch file and read the comments in the code.
Code:
REM If you get error in partition #11 keep it anyway it is still usable
REM this seem to be a normal "feature" of the nand chips used in the tablet...

Ok, I see where you found that comment, in the zip file.
Jim

Hi,
I've been doing some more detailed analysis of my dmesg.
First of all, here's snippets with info from dmesg, showing:
1) kernel boot parameters, with partition sizes
2) status of MTD partitions
mtdparts=tegra_nand:
[email protected](misc),
[email protected](recovery),
[email protected](boot),
[email protected](system),
[email protected](cache),
[email protected](bootbmp),
[email protected](logodata)
<5>[ 7.086731] 7 cmdlinepart partitions found on MTD device tegra_nand
<5>[ 7.092987] Creating 7 MTD partitions on "tegra_nand":
<5>[ 7.098130] 0x000000740000-0x000001740000 : "misc"
<5>[ 7.103663] 0x000003ce0000-0x000004ce0000 : "recovery"
<5>[ 7.109346] 0x000004d60000-0x000005d60000 : "boot"
<5>[ 7.114670] 0x000005de0000-0x0000125e0000 : "system"
<6>[ 7.119773] tegra_nand tegra_nand: Block 0x43b is bad [chip=0,offset=0x8760000]
<5>[ 7.127928] 0x000012680000-0x00001ff80000 : "cache"
<5>[ 7.133741] 0x0000017c0000-0x000001bc0000 : "bootbmp"
<5>[ 7.139340] 0x000001c40000-0x000003c40000 : "logodata"
<6>[ 7.144567] tegra_nand tegra_nand: Block 0x186 is bad [chip=0,offset=0x30c0000]
When I do nvflash --read on all the partitions, I get read errors on partition 8 and partition 11, so I tried to figure out which partition those are, from the dmesg msgs and the sizes shown:
Partition 8
Name=OGO
DeviceId=19
StartSector=12928
NumSectors=16384
BytesPerSector=2048
16834 x 2048 = 34476032 BYTES = 33,668 KBYTES ==>
ALSO:
0x000001c40000-0x000003c40000 = 2000000 = 33554432 BYTES = 32,768 KBYTES ==> THIS IS THE 'logodata' partition
[See: http://forum.xda-developers.com/archive/index.php/t-859884-p-2.html,
"Additionally, the boot animation is left over from another mod. To remove it and see the Vegan boot animation, go into /logodata and delete everything in it."]
So, it appears this bad block is occurring in the 'logodata' partition, which apparently has something to do with the boot animation.
Partition 11
Name=APP
DeviceId=19
StartSector=45696
NumSectors=102400
BytesPerSector=2048
102400 x 2048 = 209715200 BYTES = 204,800 KBYTES ==> THIS IS 'system' partition
ALSO:
0x000005de0000-0x0000125e0000 = C800000 = 209715200 ??
So, it appears this bad block is occurring in the 'system' partition, which I believe is /system.
I'm particularly concerned about the one in the 'system' partition, i.e., probably if I pull that partition using nvflash, and then later try to restore it, /system would be bad/corrupted?
Jim

Related

[TOOL] rkflashtool for Linux and rk2808, rk2818 and rk2918 based tablets

Hi,
Because I don't run Windows nor NetBSD, I rewrote rkflash from scratch with the use of libusb-1.0, so you can now read and write your rk2818-based tablet's flash memory under Linux (also w/o the need to root your tablet). Credit for reverse-engineering the protocol goes to the original author of rkflash (see source).
Small guide
- unzip the file
- compile
Linux (Debian, Ubuntu, ...)
Code:
sudo apt-get install libusb-1.0-0-dev
gcc -o rkflashtool rkflashtool.c -lusb-1.0 -O2 -W -Wall -s
Mac OS X (thanks to surfer63, binary here)
Code:
sudo port install libusb
gcc -I/opt/local/include -I/opt/local/include/libusb-1.0 \
-L/opt/local/lib -o rkflashtool rkflashtool.c -lusb-1.0 -O2 -W -Wall
Preparation
- powerdown your tablet
- disconnect all cables
To get into flash mode differs for many tablets. Google around or use trial and error
- insert the USB cable in computer
- hold vol+ (or put on/off/locked-switch in the locked position)
- insert the other end of your cable in the tablet
- wait a few seconds
- release vol+
Now if you run lsusb, the following line should appear:
Bus 001 Device 044: ID 2207:281a (290a for rk2918 based tablets)
Bus and device number may be different. The screen of your tablet stays black.
The USB device must be readable and writable for the user running rkflashtool. If that's not the case, you'll see an error like this:
Code:
$ ./rkflashtool b
libusb couldn't open USB device /dev/bus/usb/001/048: Permission denied.
libusb requires write access to USB device nodes.
rkflashtool: fatal: cannot open device
This can be fixed in several ways (chmod, run as root, udev rules) but that's beyond the scope of this posting. For now, chmod 666 the device mentioned in the error message.
Usage of rkflashtool
Code:
$ ./rkflashtool
rkflashtool: fatal: usage:
rkflashtool b reboot device
rkflashtool r offset size >file read flash
rkflashtool w offset size <file write flash
offset and size are in units of 512 bytes
On my tablet, the boot partition starts at offset 0x8000 (in blocks of 512 bytes)
Its size is 0x2000 blocks
To backup the partition, issue:
Code:
$ ./rkflashtool r 0x8000 0x2000 >boot.img.backup
rkflashtool: info: interface claimed
rkflashtool: info: reading flash memory at offset 0x00008000
rkflashtool: info: reading flash memory at offset 0x00008020
.......
rkflashtool: info: reading flash memory at offset 0x00009fe0
To write a new boot.img or an old backup back to the device:
Code:
$ ./rkflashtool w 0x8000 0x2000 <boot.img.backup
rkflashtool: info: interface claimed
rkflashtool: info: writing flash memory at offset 0x00008000
rkflashtool: info: writing flash memory at offset 0x00008020
.......
rkflashtool: info: writing flash memory at offset 0x00009fe0
You can find a list of all partitions of your tablet in the HWDEF file, which is inside the update.img for your tablet. If no such file is available, you can also look at /proc/cmdline on a running device (either through adb or a terminal app running on the device itself). Depending on the tablet, you might need root access to view /proc/cmdline. Another option is dumping the first 0x2000 blocks of nand flash by issuing rkflashtool r 0x0000 0x2000 >parm. View the file with hexedit, xxd, or a similar program. The kernel parameters contain a description of several mtd partitions (sizes and offsets).
After reading and writing at will, you can reboot your tablet by issuing
./rkflashtool b
Note that if your tablet has an on/off/locked-switch and it is still in the locked position, rebooting won't work.
If the file you are writing is smaller than the specified size, the rest is padded with zeroes. If it's bigger, it will be truncated. This is different from rkflash, which will overwrite blocks beyond the partition size.
rkflashtool does not support flashing a new bootloader directly.
If you have a different tablet, please try rkflashtool b and r first before flashing (w) something new.
Standard DISCLAIMER with regard to bricking your tablet applies.
Enjoy!
EDIT: better build instructions, clean up text
EDIT2: works on rk2918 tablets too (tested on Arnova 7 G2) if you change the USB product id from 0x281a to 0x290a before compilation
EDIT3: released version 2 of rkflashtool. now supports rk2918 tablets out of the box. if it doesn't find one, it falls back to rk2808/rk2818. also, updated the wording a bit.
EDIT4: new mac osx binary
EDIT5: more ways to find offsets and sizes of partitions on your tablet
EDIT6: small emphasis changes above and...
version 1 is here ONLY for archival purposes or if version 2 does not work on your rk28xx tablet. In all other cases, you need to download rkflashtool-v2.zip
Thanks a lot for this flash tool. I'm on MacOSX and Ubuntu and don't have Windows either. I tried the original rkflash as well but couldn't get it to work. On my Ubuntu boxes your rkflashtool compiles and works fine.
My Archos 7 HT V2 presents itself also as:
Code:
Bus 002 Device 004: ID 2207:281a
Reading partitions works fine and so does writing.
I did a quick modification of a system.img (left some files out) of my custom froyo rom and wrote it to my tablet.
That works fine. As /data is a separate partition I even have all my downloaded apps, data, settings, etc. This makes modifying a new rom much faster then building a complete update.img, flashing it, restore some data and then start testing.
Nice work.
great! finally I can remove one line from my todo list
thank you!
EDIT:
random notes (I don't see your code yet so it may be fixed, then sorry)
* I always specify b(reboot) for rk2818 tablets with my rkflash because it hanged easily if I try to write multiple times without b
* parameter file need to be converted with rkcrc -p. official RKAndroid tools flashed it 5 times with offsets. (read & check 1st 0x0-0x2000 block)
* I logged how to update bootloader, but it's complicated and I could not understand probably bootloader can be updated via misc partition. see update-script in update.img. (but not recommended/no reason to do it)
EDIT2:
there is libusb for Windows and OS X. rkflashtool may work on them.
on Windows, there is RKAndroidTool.exe (not batchupgrade). but "read" function in rkflash/rkflashtool may be useful on some case on Windows
Good to hear it works for others, too! I have not had a hanging tablet after several writes in one session, but this might depend on the tablet.
Thanks for mentioning that it should also work on other platforms supported by libusb. I'd forgotten to do that.
About using update.img to flash a new bootloader, this can be done, but if you brick the tablet by flashing a wrong/faulty bootloader, you can only unbrick it with the Windows tools
Which leads me to the question: could you send me the snooped log of updating the bootloader? Two people see more than one and perhaps we can eventually manage to do this through libusb too.
ivop said:
About using update.img to flash a new bootloader, this can be done, but if you brick the tablet by flashing a wrong/faulty bootloader, you can only unbrick it with the Windows tools
Click to expand...
Click to collapse
probably you also need a needle to short pins of NAND chip
so I don't recommend to flash bootloader
ivop said:
Which leads me to the question: could you send me the snooped log of updating the bootloader? Two people see more than one and perhaps we can eventually manage to do this through libusb too.
Click to expand...
Click to collapse
I made that log several months ago with another windows machine which is not used lately. I'm not sure log is still exist... if I find it, I'll send it to you (but please don't expect)
probably you may also get log on Windows on VM on Linux. it seems VMware has log function (refer http://vusb-analyzer.sourceforge.net/tutorial.html) or there is "usbmon" function in Linux.
actually I didn't try this way myself so it may be wrong, sorry.
I've tryed a couple of firmwares, cooking my own.
Every time after flashing, tablet shows boot animation and after few seconds display becomes dark.
My investigation led me to following:
Log shows:
Code:
ERROR/Lights(865): write_int failed to open /sys/class/backlight/rk28_button_light/brightness
in /sys/class/backlight I found symlink (rk28_bl):
rk28_bl -> ../../devices/platform/rk28_backlight/backlight/rk28_bl
Shouldn't be there another symlink named r28_button_light ?
I'm using MANTA MID001 from Poland.
fun_ said:
EDIT2:
there is libusb for Windows and OS X. rkflashtool may work on them.
Click to expand...
Click to collapse
ivop said:
Good to hear it works for others, too! I have not had a hanging tablet after several writes in one session, but this might depend on the tablet.
Click to expand...
Click to collapse
I did a couple of successive writes as well from ubuntu.
ivop said:
Thanks for mentioning that it should also work on other platforms supported by libusb. I'd forgotten to do that.
Click to expand...
Click to collapse
My main platform is OSX and I immediately added libusb. So far I have not been able to compile rkflashtool despite declaring all kind of CFLAGS, CXXFLAGS and/or LDFLAGS.
Trying a little bit more.
Could you post the compiler warnings/errors here? I might be able to help out.
ivop said:
Could you post the compiler warnings/errors here? I might be able to help out.
Click to expand...
Click to collapse
I managed to compile it. It took a lot of hurdles. I used the build environment I also use for Hugin for which I'm the OSX maintainer.
I now built a single combined 32/64bit (i386/x86_64) rkflashtool that will run on 10.4.x/10.5.x/10.6.x/10.7.x (building multi-architecture, multi-version binaries/libraries in one binary/library is possible on OSX. I'm not going to explain that here but it's a feature of OSX).
The compiled version is attached. You can also attach it to your first post if you like.
It works fine. I did some reading/writing of images without issues.
If you are on OSX and have macports installed, you can do the following to build rkflashtool.
Install libusb from Macports:
Code:
sudo port install libusb
cd into the folder where your rkflashtool.c is is and run the following command:
Code:
gcc -I/opt/local/include -I/opt/local/include/libusb-1.0 \
-L/opt/local/lib -o rkflashtool rkflashtool.c -lusb-1.0 -W -Wall
This will build rkflashtool for your native environment (OSX version, hardware and config).
--- removed the rest of the post as well as the attachments. He/She who is interested in building a complete universal distributable rkflashtool can ask via this thread ---
UPDATE: Works on rk2918 tablet too
Yesterday I have tested the tool on an Arnova 7 G2 tablet, which has an rk2918 CPU. If you change the ProductID before compilation, like this:
... libusb_open_device_with_vid_pid(c, 0x2207, 0x281a) ...
to
... libusb_open_device_with_vid_pid(c, 0x2207, 0x290a) ...
it'll work, except for rebooting the device if the tablet is still locked. To boot the tablet in bootloader mode, turn off the tablet completely, put the on/off-switch in the locked position and connect it to your computer. It should be visible now with lsusb. For further instructions, see first post. I advise dumping the first 0x2000 blocks at offset 0x0000 first as this contains the parameter block in which you can see where each partition starts and how big it is.
ivop said:
UPDATE: Works on rk2918 tablet too
Yesterday I have tested the tool on an Arnova 7 G2 tablet, which has an rk2918 CPU. If you change the ProductID before compilation, like this:
... libusb_open_device_with_vid_pid(c, 0x2207, 0x281a) ...
to
... libusb_open_device_with_vid_pid(c, 0x2207, 0x290a) ...
Click to expand...
Click to collapse
Feature request :
I's nice but could you also make it a startup option, like the b,r,w options, with an if-else option in the source code? Something like (RK)2818 and (RK)2918 and maybe even for the older ones: (RK)2808.
In that case you only need one binary. Users who are going to use the tool will definitely know what CPU they have.
surfer63 said:
Feature request :
I's nice but could you also make it a startup option, like the b,r,w options, with an if-else option in the source code? Something like (RK)2818 and (RK)2918 and maybe even for the older ones: (RK)2808.
In that case you only need one binary. Users who are going to use the tool will definitely know what CPU they have.
Click to expand...
Click to collapse
I released a new version and updated the first post. It now tries to connect to an rk2918 tablet and if it doesn't find one, it falls back to rk2818.
The V2 version works fine too on MacOSX. The compilation is still the same for a "my machine only" version.
I compiled a universal Intel 32bit/64bit 10.4/10.5/10.6/10.7 V2 version as well.
See attached.
Note: I don't have a RK2918 so I can only test for a RK2818 tablet.
Hi,
Thanks for your thread it's very intersting.
I succeed flashing my boot partition with your tool but I don't success in remount,rw my system partition. It's cramFS and in init.rk28board.rc you can see those line :
Code:
# Mount /system rw first to give the filesystem a chance to save a checkpoint
mount cramfs [email protected] /system
mount cramfs [email protected] /system ro remount
I tried everything like replacing ro by rw, deleting the second line but my system stills in ReadOnly, don't understand why. I also tried deleting those lines to test if my flash process works properly and it's worked... So I'm lost. Any idea ?
----
Other thing, if I want to do same as flashing boot partition but with system partition is it possible with the same process ? Unfortunately I don't know the beginning offset of the partition. I don't know where to find HWDEF file. The size of partition is 00038000 (hex) bytes => 229376 (dec) bytes
Here is my /proc/mtd :
Code:
dev: size erasesize name
mtd0: 00002000 00000010 "misc"
mtd1: 00004000 00000010 "kernel"
mtd2: 00002000 00000010 "boot"
mtd3: 00004000 00000010 "recovery"
mtd4: 00038000 00000010 "system"
mtd5: 0003a000 00000010 "backup"
mtd6: 0003a000 00000010 "cache"
mtd7: 00080000 00000010 "userdata"
mtd8: 00534000 00000010 "user"
mtd9: 00020000 00000010 "pagecache"
mtd10: 00020000 00000010 "swap"
Thank you for your great job
My problem is solved. I was searching for a while but ivop gave the answer in a previous post
I advise dumping the first 0x2000 blocks at offset 0x0000 first as this contains the parameter block in which you can see where each partition starts and how big it is.
Click to expand...
Click to collapse
So I did it, after I opened an Hex Editor like GHex on Ubuntu and I can saw this :
Code:
[email protected](misc),
[email protected](kernel),
[email protected](boot),
[email protected](recovery),
[email protected](system),
[email protected](backup),
[email protected](cache),
[email protected](userdata),
[email protected](user)
So system partition starts at E000 and has a length of 38000 (hex) bytes.
Thanks for your help this thread is now in my bookmarks
And really nice job with this flashtool
I pushed latest my rkutils to https://github.com/naobsd/rkutils
rkunpack can unpack RKFW image used in RK2918 ROM, RKAF image (update.img), KRNL/PARM image used in some single partition image. unpack will be done recursively.
rkcrc can make KRNL/PARM images with -k/-p.
rkafpack can make RKAF image. (I need to write docs/howtos...)
little off-topic,
latest RK2918 ROMs which is based on "SDK2.0", new format for boot.img/recovery.img is introduced. it's almost same as common boot.img format for android. unpackbootimg/mkbootimg can be used to unpack/repack it with one exception...
there is SHA1 hash value in header of boot.img (offset 0x240 bytes). Rockchip changes it by some unknown way. normal mkbootimg can't generate same hash value as Rockchip, so we can't make custom boot.img with new format
fortunately, we can split new boot.img, and we can make separate kernel.img and boot.img(ramdisk) like as pre-SDK2.0 RK2918 ROMs, which is loadable with new bootloader in SDK2.0 ROMs.
--
btw I just found interesting one: https://github.com/jhonxie/rk2918_tools
relsyou said:
My problem is solved. I was searching for a while but ivop gave the answer in a previous post
So I did it, after I opened an Hex Editor like GHex on Ubuntu and I can saw this :
Code:
[email protected](misc),
[email protected](kernel),
[email protected](boot),
[email protected](recovery),
[email protected](system),
[email protected](backup),
[email protected](cache),
[email protected](userdata),
[email protected](user)
So system partition starts at E000 and has a length of 38000 (hex) bytes.
Thanks for your help this thread is now in my bookmarks
And really nice job with this flashtool
Click to expand...
Click to collapse
I'll add that to my first post. Also, you can view /proc/cmdline to see a list of partitions. It's part of the kernel command line.
Note that the lengths are not in bytes but in blocks of 512 bytes. This happens to be the same as the requirements of the rkflashtool btw (length in blocks).
As for having a writable system partition, currently the system partition is cramfs which cannot be written to. Ever. If you want a writable system partition, you need to change it to ext3 for example. That means unpacking fun_'s system.img and recreating it as an ext3 partition.
In short:
Unpack cramfs img with cramfsck -x (as root, so you preserve permissions and uid/gid)
Create an empty file the size of your system partition (dd if=/dev/null of=fubar.img bs=512 count=...... et cetera, do the math)
mkfs.ext3 fubar.img
mount -o loop fubar.img /someplacemountable
copy contents of old image to /someplacemountable (use cp -a to preserve ownership etc)
umount
flash fubar.img to system partition
change init.rk28board.rc to reflect the changes
reflash boot.img
reboot device
This is untested, but should work in theory.
Another option is to keep the system partition read-only and use unionfs to overlay a writable partition. I'm not sure if this can be a file on your userdata partition mounted with -o loop, but I suppose it can. This depends on your kernel having unionfs and loopback support though.
fun_ said:
I pushed latest my rkutils to https://github.com/naobsd/rkutils
Click to expand...
Click to collapse
Nice! I was thinking about creating an rkpack(tool ) myself, but I see it's not necessary anymore.
here is an example for rkafpack
Code:
$ rkunpack N3NET-2.3-20110722.img
[COLOR="Red"][B]FIRMWARE_VER:1.0.0[/B][/COLOR]
[COLOR="Red"][B]MACHINE_MODEL:rk2818sdk[/B][/COLOR]
MACHINE_ID:
[COLOR="Red"][B]MANUFACTURER:rock-chips[/B][/COLOR]
unpacking 12 files
-------------------------------------------------------------------------------
00000800-00000fff [COLOR="Red"][B]HWDEF:HWDEF[/B][/COLOR] 797 bytes
00001000-000017ff [COLOR="Red"][B]package-file:package-file[/B][/COLOR] 532 bytes
00001800-00021fff [COLOR="Red"][B]bootloader:RK28xxLoader(L).bin[/B][/COLOR] 131700 bytes
00022000-000227ff [COLOR="Red"][B]parameter:parameter:[email protected][/B][/COLOR] 506 bytes
00022800-0002e7ff [COLOR="Red"][B]misc:Image/misc.img:[email protected][/B][/COLOR] 49152 bytes
0002e800-0066bfff [COLOR="Red"][B]kernel:Image/kernel.img:[email protected][/B][/COLOR] 6541946 bytes
0066c000-006947ff [COLOR="Red"][B]boot:Image/boot.img:[email protected][/B][/COLOR] 163844 bytes
00694800-008e8fff [COLOR="Red"][B]recovery:Image/recovery.img:[email protected][/B][/COLOR] 2441220 bytes
008e9000-085fc7ff [COLOR="Red"][B]system:Image/system.img:[email protected][/B][/COLOR] 131149828 bytes
----------------- [COLOR="Red"][B]backup:SELF:[email protected][/B][/COLOR] (N3NET-2.3-20110722.img) 140498948 bytes
085fc800-085fcfff [COLOR="Red"][COLOR="Red"][B]update-script:update-script[/B][/COLOR][/COLOR] 1294 bytes
085fd000-085fd7ff [COLOR="Red"][B]recover-script:recover-script[/B][/COLOR] 266 bytes
-------------------------------------------------------------------------------
unpacked
$ rkafpack \
[COLOR="Red"][B]FIRMWARE_VER:1.0.0[/B][/COLOR] \
[COLOR="Red"][B]MACHINE_MODEL:rk2818sdk[/B][/COLOR] \
[COLOR="Red"][B]MANUFACTURER:rock-chips[/B][/COLOR] \
[COLOR="Red"][B]HWDEF:HWDEF[/B][/COLOR] \
[COLOR="Red"][B]package-file:package-file[/B][/COLOR] \
'[COLOR="Red"][B]bootloader:RK28xxLoader(L).bin[/B][/COLOR]' \
[COLOR="Red"][B]parameter:parameter:[email protected][/B][/COLOR] \
[COLOR="Red"][B]misc:Image/misc.img:[email protected][/B][/COLOR] \
[COLOR="Red"][B][B]kernel:Image/kernel.img:[email protected][/B][/B][/COLOR] \
[COLOR="Red"][B]boot:Image/boot.img:[email protected][/B][/COLOR] \
[COLOR="Red"][B]recovery:Image/recovery.img:[email protected][/B][/COLOR] \
[COLOR="Red"][B]system:Image/system.img:[email protected][/B][/COLOR] \
[COLOR="Red"][B]backup:SELF:[email protected][/B][/COLOR] \
[COLOR="Red"][B]update-script:update-script[/B][/COLOR] \
[COLOR="Red"][B]recover-script:recover-script[/B][/COLOR] \
> new.img
$ sha1sum N3NET-2.3-20110722.img new.img
e758a6c47dca7f09f0b8a82ad89b0cd7c7c8e826 N3NET-2.3-20110722.img
e758a6c47dca7f09f0b8a82ad89b0cd7c7c8e826 new.img
some values are empty in RK2818 ROM.
--
here is how to make RKFW image
Code:
$ rkunpack N50-2.3-20111103-ZZ-SDK2.0.img
VERSION:2.0.3
unpacking
00000000-00000065 N50-2.3-20111103-ZZ-SDK2.0.img-HEAD 102 bytes
00000066-00022623 N50-2.3-20111103-ZZ-SDK2.0.img-BOOT 140734 bytes
00022624-0c342627 update.img 204603396 bytes
unpacking update.img
================================================================================
FIRMWARE_VER:0.2.3
MACHINE_MODEL:rk29sdk
MACHINE_ID:007
MANUFACTURER:RK29SDK
unpacking 10 files
-------------------------------------------------------------------------------
00000800-00000fff package-file:package-file 540 bytes
00001000-000237ff bootloader:RK29xxLoader(L)_V2.08.bin 140734 bytes
00023800-00023fff parameter:parameter:[email protected] 610 bytes
00024000-0002ffff misc:Image/misc.img:[email protected] 49152 bytes
00030000-006a3fff boot:Image/boot.img:[email protected] 6766592 bytes
006a4000-01167fff recovery:Image/recovery.img:[email protected] 11288576 bytes
01168000-0c31efff system:Image/system.img:[email protected] 186346496 bytes
----------------- backup:SELF:[email protected] (update.img) 204603396 bytes
0c31f000-0c31f7ff update-script:update-script 933 bytes
0c31f800-0c31ffff recover-script:recover-script 266 bytes
-------------------------------------------------------------------------------
================================================================================
00022624-0c342627 N50-2.3-20111103-ZZ-SDK2.0.img-MD5 32 bytes
unpacked
$ cat N50-2.3-20111103-ZZ-SDK2.0.img-HEAD N50-2.3-20111103-ZZ-SDK2.0.img-BOOT update.img > new.img
$ md5sum new.img
[COLOR="Red"][B]5191abc65649eacf8d2476e37d84a046[/B][/COLOR] new.img
$ cat N50-2.3-20111103-ZZ-SDK2.0.img-MD5
5191abc65649eacf8d2476e37d84a046
$ echo -n [COLOR="Red"][B]5191abc65649eacf8d2476e37d84a046[/B][/COLOR] >> new.img
$ sha1sum N50-2.3-20111103-ZZ-SDK2.0.img new.img
3120b13df8886e0ddfae0e35379443c27c925572 N50-2.3-20111103-ZZ-SDK2.0.img
3120b13df8886e0ddfae0e35379443c27c925572 new.img

TF700T complete flash layout

I spent some time in analyzing of flash layout. The comprehensive description below attempts to map each byte of the flash and describes way to extract it.
I would be glad if somebody could provide more detailed info about bootloader, signatures, DRM etc.
Patches are welcome.
Code:
mmcblk0 layout
All dumps were done on Asus Eee Pad Transformer Infinity TF700T, 64GB version, firmware 9.4.5.26, locked
mmcblk0 off-partition section
Offset: 0 (0x0)
Size: 38273024 (0x2480000)
Read command: busybox dd if=/dev/block/mmcblk0 of=/mnt/sdcard/mmcblk0pre1.img bs=524288 count=73
Offset: 0 (0x0)
Size: 3670016 (0x380000)
Contains: Zeroes
Purpose: Unknown
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s1.img bs=3670016 count=1
Process command: tr -d '\0' <mmcblk0pre1s1.img >mmcblk0pre1s1nz.img # mmcblk0pre1s1nz.img must be empty file
Offset: 3670016 (0x380000)
Contains: Recovery kernel image followed by zeroes
Size: 8388608 (0x800000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s2.img bs=524288 skip=7 count=16
Process commands:
perl split_bootimg.pl mmcblk0pre1s2.img
mkdir mmcblk0pre1s2.img-ramdisk
cd mmcblk0pre1s2.img-ramdisk
zcat ../mmcblk0pre1s2.img-ramdisk.gz | cpio -i
cd ..
# end Process commands
Offset: 12058624 (0xb80000)
Contains: Regular boot kernel image followed by zeroes
Size: 8388608 (0x800000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s3.img bs=524288 skip=23 count=16
Process commands:
perl split_bootimg.pl mmcblk0pre1s3.img
mkdir mmcblk0pre1s3.img-ramdisk
cd mmcblk0pre1s3.img-ramdisk
zcat ../mmcblk0pre1s3.img-ramdisk.gz | cpio -i
cd ..
# end Process commands
Offset: 20447232 (0x1380000)
Contains: Block of 16 bytes followed by 0x2de0 hexadecimal numbers followed by FF
Size: 12288 (0x3000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s4.img bs=524288 skip=39
Vital data:
Extract command: dd if=mmcblk0pre1s4.img of=mmcblk0pre1s4ss2.img bs=4096 skip=3
Binary part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch1.img bs=16 count=1
Hexadecimal part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch2.img bs=16 count=734 skip=1
Process command: unhex <mmcblk0pre1s4ss1ch2.img >mmcblk0pre1s4ss1ch2bin.img
FF part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch3.img bs=16 skip=735
Process command: tr -d '\377' <mmcblk0pre1s4ss1ch3.img >mmcblk0pre1s4ss1ch3nff.img # mmcblk0pre1s4ss1ch3nff.img must be empty file
Zeroes:
Extract command: dd if=mmcblk0pre1s4.img of=mmcblk0pre1s4ss1.img bs=4096 count=3
Process command: tr -d '\0' <mmcblk0pre1s4ss2.img >mmcblk0pre1s4ss2nz.img # mmcblk0pre1s4ss2nz.img must be empty file
Purpose: Probably encrypted bootloader
mmcblk0p1
Offset: 38273024 (0x2480000)
Size: 805306368 (0x30000000)
File system size: 196608 * 4096 = 805306368 (fully occupies partition)
Format: Linux ext4 filesystem
Mounted at: /system
Mount options: read only, extended attributes, ACL
Permissions: only root can manipulate
Contains: Base system and embedded applications
Purpose: Base system
mmcblk0p2
Offset: 843579392 (0x32480000)
Size: 448790528 (0x1ac00000)
File system size: 109568 * 4096 = 448790528 (fully occupies partition)
Format: Linux ext4 filesystem
Mounted at: /cache
Mount options: read/write, no SUID, no device nodes, no atime
Permissions: only root can manipulate, UID system and GID cache can read and write
Contains: Cache
Purpose: Application cache
Note: The volume has the same UUID as mmcblk0p1
mmcblk0p3
Offset: 1292369920 (0x4d080000)
Size: 2097152 (0x200000)
File system size: 512 * 4096 = 2097152 (fully occupies partition)
Linux rev 1.0 ext3 filesystem
Not mounted
Permissions: GID system can manipulate
Contains: Empty file system
Purpose: Recovery /misc
Referenced by: /system/lib/libandroid_runtime.so recovery ramdisk: /etc/recovery.fstab
Note: File system is referenced in recovery as emmc, not ext3!
mmcblk0p4
Offset: 1294467072 (0x4d280000)
Size: 855638016 (0x33000000)
File system size: 208896 * 4096 = 855638016
Linux rev 1.0 ext3 filesystem
Not mounted
Permissions: GID system can manipulate
Contains: Empty file system
Purpose: Recovery /staging
Referenced by: recovery ramdisk: init.rc /etc/recovery.fstab
mmcblk0p5
Offset: 2150105088 (0x80280000)
Size: 5242880 (0x500000)
File system size: 5092 * 1024 = 5147488
Format: FAT32 file system, no partition table, MS-DOS "Non-system disk" boot block
Not mounted
Permissions: only root can manipulate
Contains: File system with files:
Serial numbers (ISN, PPID, SSN, UUID)
Calibration data (AL3010 light sensor, AMI304 magnetic sensor, KXTF9 motion sensor)
Purpose: Device specific unique system data, mounted as /btmac during Android boot
Referenced by: /system/bin/wifimacwriter /system/bin/brcm_patchram_plus /system/bin/sensors-config /system/bin/sixpair ramdisk: /init recovery ramdisk: /etc/recovery.fstab /init
mmcblk0p5 off file-system area
Offset in section: 5147488 (0x4e8b60)
Size: 28672 (0x7000)
Read command: busybox dd if=/dev/block/mmcblk0p5 of=/mnt/sdcard/mmcblk0p5s2.img bs=1024 skip=5092
Process command: tr -d '\0' <mmcblk0p5s2.img >mmcblk0p5s2nz.img # mmcblk0p5s2nz.img must be empty file
mmcblk0p6
Offset: 2155347968 (0x80780000)
Size: 524288 (0x80000)
Format: binary data
Permissions: UID drm can manipulate
Contains: 208 bytes of binary data, the rest are zeroes
Purpose: DRM, probably contains encrypted DRM key
Referenced by: /system/bin/wvdrmserver /system/vendor/lib/drm/libdrmwvmplugin.so
mmcblk0p7
Offset: 2155872256 (0x80800000)
Size: 5242880 (0x500000)
Format: empty
Contains: Zeroes
Purpose: Unknown
mmcblk0p8
Offset: 2161115136 (0x80d00000)
Size: 61415620608 (0xe4ca80000)
File system size: 14994040 * 4096 = 61415587840
Format: Linux ext4 filesystem
Mounted at: /data
Mount options: read/write, no SUID, no device nodes, no atime
Permissions: only root can manipulate, read and write are directory specific
Contains: User applications, user data, and virtual internal SD card
Note: /data/media is mounted via UID/GID stripping FUSE as /mnt/sdcard
mmcblk0p8 off file-system area
Offset in section: 61415587840 (0xe4ca78000)
Size: 32768 (0x8000)
Read command: busybox dd if=/dev/block/mmcblk0p8 of=/mnt/sdcard/mmcblk0p8s2.img bs=4096 skip=14994040
mmcblk0 off-partition section
Offset: 63576735744 (0xecd780000)
Size: 524288 (0x80000)
Read command: busybox dd if=/dev/block/mmcblk0 of=/mnt/sdcard/mmcblk0post8.img bs=524288 skip=121263
Process command: tr -d '\0' <mmcblk0p8s2.img >mmcblk0p8s2nz.img # mmcblk0p8s2nz.img must be empty file
Offset: 63576735744 (0xecd780000)
Offset in section: 0 (0x0)
Size: 507392 (0x7be00)
Contains: Zeroes
Purpose: Unknown
Extract command: dd if=mmcblk0post8.img of=mmcblk0post8s1.img bs=507392 count=1
Process command: tr -d '\0' <mmcblk0post8s1.img >mmcblk0post8s1nz.img # mmcblk0post8s1nz.img must be empty file
Offset: 63577243136 (0xecd7fbe00)
Offset in section: 507392 (0x7be00)
Size: 16896 (0x4200)
Contains: EFI Partition table (partition names: APP, CAC, MSC, USP, PER, YTU, CRA, UDA)
Extract command: dd if=mmcblk0post8.img of=mmcblk0post8s2.img bs=512 skip=991
Purpose: Partition table
Total size of mmcblk0: 63577260032 (0xecd800000)
Notes:
can manipulate = can read, write partition vital data, only root can mount
can read, write = can read, write partition file system contents
Read commands are ran on the Transformer
Extract and process commands are run anywhere, with pre-read image file in the current directory.
You need dd with large files support. Vanilla dd on TF700T does not support large files. Busybox dd does.
Dropbox link to Asus_Transformer_Infinity_TF700T/flash_layout.txt
Wow, thanks for this detailed analysis - much more detailed than mine.
So what can I add to your research?
Tegra-based systems have another partition table, which has a proprietary layout and an unknown purpose (maybe just important for NVFlash and for flashing blobs?). Looking at the flash.cfg in the NVFlash package from AndroidRoot.mobi, we can get the Tegra partition layout and partition names:
Partition number 1 is missing in the list, maybe it contains the extremely well-hidden APX mode recovery code or even the answer to life, the universe and everything.
The following 3 partitions are located at the beginning of mmcblk0 and their contents are apparently encrypted with a device-specific key. For some reason, with ICS-based ROMs it reads as all zeros; in JB-based ROMs additional mmcblk0boot0 and mmcblk0boot1 partitions appear which together cover this area. The "bricksafe.img" in the nvflash guide covers these 3 partitions.
2 BCT: Tegra Boot Configuration Table - 3145728 bytes
3 PT: Tegra Partition Table - 524288 bytes
4 EBT: Bootloader - 8388608 bytes
You already know the following 2:
5 SOS: Recovery kernel - 8388608 bytes
6 LNX: Linux kernel - 8388608 bytes
Then some more funny ones:
7 CER: I think this stands for "Certificate" and contains the bootloader unlock token. - 8388608 bytes. If I calculated correctly, this is at 0x1380000 into mmcblk0. Saved as "unlock-token.img" in the nvflash guide.
8 IMG: no idea what this is for - 8388608 bytes
9 GP1: space for a GPT partition table, maybe unused - 1048576 bytes
Now the regular partitions follow (p1 to p8):
10 APP: p1 = /system (Android OS)
11 CAC: p2 = /cache (for communication between Android and recovery)
12 MSC: p3 ="misc", whatever that is. On the TF101 it was used for bootloader commands.
13 USP: p4 = The update staging partition. Update blobs are copied here and flashed to the correct partition by the bootloader.
14 PER: p5 = device-specific config in a FAT filesystem
15 YTU: p6 = Apparently the DRM key. Confirmed to be overwritten with 0 by the unlocking process.
16 CRA: p7 = unknown (reserved for crash dumps?)
17 UDA: p8 = /data (Android user data)
And finally:
18 GPT: the EFI partition table that is actually used by the kernel
Well, it seems, that something (ICS stock kernel, hardware) hides contents of the first (at most) 0x380000 bytes of flash.
I am locked, and I have some token at 0x1380000 as well.
I am still thinking about a way to unlock, keep access to nvflash, and upgrade to JB keeping DRM working, even at cost of using stock system. That is why I wanted to backup and analyze everything and find all keys and signatures.
It would be also nice to know, whether there are areas of flash with hardware or kernel write lock.
utx said:
Well, it seems, that something (ICS stock kernel, hardware) hides contents of the first (at most) 0x380000 bytes of flash.
I am locked, and I have some token at 0x1380000 as well.
Click to expand...
Click to collapse
Yes, before unlocking I had something very similar to you there - a 16 byte header followed by some hexdump. I don't know what it was. It was overwritten by the unlock process with a 4 byte data block prefixed with a "-SIGNED-BY-SIGNBLOB-" header and followed by 256 bytes of what looks like a digital signature, very similar to the signed update blobs.
utx said:
I am still thinking about a way to unlock, keep access to nvflash, and upgrade to JB keeping DRM working, even at cost of using stock system. That is why I wanted to backup and analyze everything and find all keys and signatures.
Click to expand...
Click to collapse
Definitely back up the YTU partition before unlocking (p6) and then make the nvflash backups - but maybe the key must match something that is broken by the unlocking process, or it is renewed periodically, etc., so it might not help. Maybe try using DRM before unlocking and watch if the content of the partition changes over time.
utx said:
It would be also nice to know, whether there are areas of flash with hardware or kernel write lock.
Click to expand...
Click to collapse
Never tried to write directly to the block device - too scared to break something.
---------- Post added at 09:32 PM ---------- Previous post was at 09:28 PM ----------
Another small addition:
Note: /data/media is mounted via UID/GID stripping FUSE as /mnt/sdcard
Click to expand...
Click to collapse
This FUSE trick also makes /mnt/sdcard case-insensitive.
I just thought of something. What if you launched a data recovery process and recovered the DRM keys for the device?
ostar2 said:
I just thought of something. What if you launched a data recovery process and recovered the DRM keys for the device?
Click to expand...
Click to collapse
How do you define "data recovery process"? You cannot recover data that has been overwritten.
_that said:
How do you define "data recovery process"? You cannot recover data that has been overwritten.
Click to expand...
Click to collapse
Well, if the DRM partition is write enabled, it may be possible to restore its contents, if you backed it up before unlock (it is probably per-device unique). But it could be insufficient. Locked bootloader can be different than unlocked bootloader, and may drop cipher needed for DRM decihering. It is just a theory. Somebody could proof it or falsify, if:
1) Backed all accessible data before unlock.
2) Unlocked (and to be safe, also made brickproof image).
3) Recovered the data creates in step 1.
Will DRM work then? Or did we need the contents of (currently inaccessible) locked stock data of the first megabytes?
But I see no way, how to back-up first megabytes of locked device (on ICS; JB is not as interesting for us, once you upgrade to JB, you cannot create brickproof image for nvflash).
I even don't know, which part of the subsystem causes these megabytes being reported as zeroes. Is it stock Asus ICS kernel? Is it bootloader? Is it a hardware lock on the flash device?
Good idea, but what I meant by "Data Recovey". Is restoring the deleted data from that filesystem/partition.
ostar2 said:
Good idea, but what I meant by "Data Recovey". Is restoring the deleted data from that filesystem/partition.
Click to expand...
Click to collapse
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
_that said:
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Click to expand...
Click to collapse
To install AndroidRoot bootloader and by that getting nvflash blobs, you have to unlock first... The order of your steps is therefore wrong.
firetech said:
To install AndroidRoot bootloader and by that getting nvflash blobs, you have to unlock first... The order of your steps is therefore wrong.
Click to expand...
Click to collapse
Oops, thanks for noticing. I edited my post.
what if we were to read from the NAND externally (RAW)....xbox 360 style...wouldn't that be the same as nvflash....
except that the three partitions in question are encrypted with a key that is probably unique per Tegra...
2 BCT: Tegra Boot Configuration Table - 3145728 bytes
3 PT: Tegra Partition Table - 524288 bytes
4 EBT: Bootloader - 8388608 bytes
but I would suppose it wouldn't be a problem since a raw flash would restore everything back to normal...even if we can't read it..the CPU can..and that's all that matters.
---------- Post added at 11:21 AM ---------- Previous post was at 11:13 AM ----------
never mind...its a BGA
_that said:
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Click to expand...
Click to collapse
Correct order maybe.
- get new TF700
- update to 9.4.5.26.
- root it using debugfs
- make a backup of /dev/block/*.*
- unlock (this erases mmcblk06 and voids warranty)
- install AndroidRoot hacked bootloader to make blobs for nvflash
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Q1:If i backed up 9.4.5.26 all block image.After i updated 9.4.5.30 can i get the nvflash blob from backed up images?No way to dig out the blob key from the backup?
W3ber said:
Q1:If i backed up 9.4.5.26 all block image.After i updated 9.4.5.30 can i get the nvflash blob from backed up images?No way to dig out the blob key from the backup?
Click to expand...
Click to collapse
No way - the BCT, bootloader, etc. is not visible to the kernel at all (so it's not included in your images), and I don't know which kind of magic the blob creation tool uses, but I assume it's more than reading stuff from the nand.

[DISCUSSION]All about boot.img's and kernels!!!

So, I've seen many a people talk about boot.imgs, and kernels, and mostly spamming dev threads. So, why not create a new thread for it?
Here, ask your questions related to kernels and boot.imgs, and feel free to post any *improvements* that you've made to an existing boot.img/kernel, and so...
Also, feel free to join the discussion, feature requests, whether or not possible, etc, etc.
This thread might be one whole lot of junk, but still if it helps from keeping people from spamming dev threads, why not
So, start!
Lets start from the basics...
Compiling a kernel is easy, IMO :fingers-crossed:
But, the unpacking repacking stuff is a bit difficult...
Anyways, here's the basics:
Packaging kernel for flashing on target device (by sakindia123): http://forum.xda-developers.com/showpost.php?p=31656992&postcount=3
Packaging kernel for flashing on target device (by #define): http://forum.xda-developers.com/showthread.php?t=2114594 (start from step 6)
New CM11 Ramdisk changes...
Well, this was taking up a lot of space on the main thread... Read it here :fingers-crossed:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
edit: some more thoughts...
If you're going to boot this ROM from your SD Card, then, better beware of the new changes in the mounting process... A new fstab.pico is handling the mounting processes..
And, its moved from init.rc to init.pico.rc :|
So, If you're gonna do the editing stuff, you'd find that init.pico.rc has:
Code:
on fs
# mount mtd partitions
# Mount /system rw first to give the filesystem a chance to save a checkpoint
mount yaffs2 [email protected] /system
mount yaffs2 [email protected] /system ro remount
mount yaffs2 [email protected] /data nosuid nodev
mount yaffs2 [email protected] /cache nosuid nodev
# mount partitions
mount_all /fstab.pico
So, if you're gonna just edit those lines, and just change the [email protected], or the [email protected] lines, then, you'd likely be pretty much booting the ROM again from your internal nand. the main reason for this being the line below: mount_all /fstab.pico, which is the fstab.pico I was talking about...
Now, if you'd open up fstab.pico, its pretty much like the /etc/fstab that you'd find on any linux system. the typical device, mountpoint, type and all... If you'd open it, you'll find this:
Code:
#<src> <mnt_point> <type> <mnt_flags and options> <fs_mgr_flags>
/dev/block/mtdblock3 /system yaffs2 ro,barrier=1 wait
/dev/block/mtdblock4 /cache yaffs2 nosuid,nodev,barrier=1 wait,check
/dev/block/mtdblock5 /data yaffs2 nosuid,nodev,noauto_da_alloc,barrier=1 wait,check
/devices/platform/msm_sdcc.1/mmc_host/mmc0 auto vfat defaults voldmanaged=sdcard0:auto
Well, those were the lines from the CM11 preview, not exactly the same, the last line was derped by me in my vain attempts to get the sdcard running... Oh, and the mounting of the SD Card's completely changed too :silly:
BTW, getting back to mounting the ROM from the sdcard, you'd need to edit this fstab.pico file, to something similar to this:
Code:
#<src> <mnt_point> <type> <mnt_flags and options> <fs_mgr_flags>
/dev/block/mmcblk0p2 /system ext4 ro,barrier=1 wait
/dev/block/mmcblk0p4 /cache ext4 nosuid,nodev,barrier=1 wait,check
/dev/block/mmcblk0p3 /data ext4 nosuid,nodev,noauto_da_alloc,barrier=1 wait,check
/devices/platform/msm_sdcc.1/mmc_host/mmc0 auto vfat defaults voldmanaged=sdcard0:auto
Well this is the first post, asking for help... Hope you get the hint *wink *wink
In case you can't get something done, ask for help here!
I tried porting the touchscreen gestures from Siyah Kernel with reference from this commit: https://github.com/gokhanmoral/siyahkernel3/commit/9f57d9efc7458c1a9f540cd04bc5cb14e08fb342
Well, that more or less turned out to be like this: https://github.com/vineethraj49/android_kernel_htc_pico/tree/gestures (check the last few commits...)
And, turns out it works :laugh: with a small bug
1. only single finger gestures work, i think....
2. no way to get the infinite while loop started at init.d
Here's how I did it... Reference thread: http://forum.xda-developers.com/showthread.php?t=1831254
the init.d script is as belows, doesn't work...
Code:
#!/system/bin/sh
echo "
# Gesture 1 - swipe 1 finger near the top
1:1:(0|150,0|150)
1:1:(210|320,0|150)
" > /sys/devices/virtual/misc/touch_gestures/gesture_patterns
while [ 1 ]
do
GESTURE=`cat /sys/devices/virtual/misc/touch_gestures/wait_for_gesture`
if [ "$GESTURE" == "1" ]; then
screencap > /sdcard/`date +%H%M%S.png`
fi;
done
So, inputted those commands using adb, in adb shell, and it works... Got a hell lot of screenshots... but, any way to fix the bugs I mentioned?
Hey, one question
I'd like to know about one thing, I've seen people talking about memory increasing kernels and kernels that mount certain partitions(like cache, data etc. etc. ) . i want to know how this works? I mean, what all things are to be done in both things for them to work?
I'll be glad if you tell this to me in a noob-friendly way. :victory:
Thank You! :fingers-crossed:
#Superuser said:
Hey, one question
I'd like to know about one thing, I've seen people talking about memory increasing kernels and kernels that mount certain partitions(like cache, data etc. etc. ) . i want to know how this works? I mean, what all things are to be done in both things for them to work?
I'll be glad if you tell this to me in a noob-friendly way. :victory:
Thank You! :fingers-crossed:
Click to expand...
Click to collapse
Note: memory increasing kernels and, kernels that mount certain partitions(like cache, data etc. etc. ) both do the exact same thing.
Clarifications:
1. There's no such thing as a memory increasing kernel, and there can't be. No!? Why not? Because, however much memory's present, so, remains. This particular *myth* comes from the slang "memory increasing scripts", i.e. scripts that mount an external SD Card's partition as the internal /data partition.
2. The kernel doesn't increase the memory (check clarification 1). The increasing in memory is done by reverse-mounting (yes, reverse-mounting partitions, is a fairly popular slang for this practice) partitions from the SDCard as the internal partitions, and thus, the phone thinks that it has a more storage, than its own internal storage.
So, how does the memory increasing get done?
Ramdisks! These are the files that go on form the root file system. The kernel itself is packed with the ramdisk. A typical android bootup sequence is this: (thanks to the writers of this article here: http://elinux.org/Android_Booting)
1. The first program which runs on any Android system is the bootloader. Technically, the bootloader is outside the realm of Android itself, and is used to do very low-level system initialization, before loading the Linux kernel. The kernel then does the bulk of hardware, driver and file system initialization, before starting up the user-space programs and applications that make up Android.
2. 'init'
A key component of the Android bootup sequence is the program 'init', which is a specialized program for initializing elements of the Android system. Unlike other Linux systems (embedded or otherwise), Android uses its own initialization program. (Linux desktop systems have historically used some combination of /etc/inittab and sysV init levels - e.g. /etc/rc.d/init.d with symlinks in /etc/rc.d/rc.). Some embedded Linux systems use simplified forms of these -- such as the init program included in busybox, which processes a limited form of /etc/inittab, or a direct invocation of a shell script or small program to do fixed initialization steps.
The Android 'init' program processes two files, executing the commands it finds in them, called 'init.rc' and 'init.<machine_name>.rc', where <machine_name> is the name of the hardware that Android is running on. (Usually, this is a code word. The name of the HTC1 hardware for the ADP1 is 'trout', and the name of the emulator is 'goldfish'.
The 'init.rc' file is intended to provide the generic initialization instructions, while the 'init.<machine_name>.rc' file is intended to provide the machine-specific initialization instructions.
====================================================================================================
Now, that's the general booting process. Now, lets look into our phone's booting process. The bootloader is "HBOOT" specialised for our phones, made by HTC.
This boots up the hardware, loads the kernel, and, the "init" process starts running. If you unpack any kernel you'd find these two files:
Code:
init.rc
init.pico.rc
As you'd have guessed by now, the init.rc is general instructions, and init.pico.rc is the hardware device specific parts.
The init process is what will set up all native services and this is similar to a regular Linux system boot.
So, the init process is also the reason why the filesystems are mounted. If you'd open up any init.rc file, you'd find these lines:
Code:
on fs
# mount mtd partitions
# Mount /system rw first to give the filesystem a chance to save a checkpoint
mount yaffs2 [email protected] /system
mount yaffs2 [email protected] /system ro remount
mount yaffs2 [email protected] /data nosuid nodev
mount yaffs2 [email protected] /cache nosuid nodev
This language is commonly known as the "Android Init Language", and you can look up for help here: https://android.googlesource.com/platform/system/core/+/master/init/readme.txt
So, this is where the filesystems get mounted.
Now, to make a reverse-mounting boot.img, we'd need to modify these lines. Lets mount the second partition in the SD Card in the data partition. So, we'd simply replace this line:
Code:
mount yaffs2 [email protected] /data nosuid nodev
with a modified line, like this:
Code:
mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev
If you'd see, this follows this configuration:
mount <type> <device> <dir> [ <mountoption> ]*
Now, the nosuid, nodev, etc are mountoptions. There are a variety of mount options, and the right choice of them, is likely to make your mounted partition accessible faster. Example: You can disable journaling, or use a different method of journaling, so that, you get access to the partitions faster, and as a direct result, your phone *might* become faster. The general mount options can be found here: http://linux.die.net/man/8/mount
So, that's about how you do the "reverse-mounting" thing.
p.s. here's the mount config that I usually use. Also posting the reasons
p.p.s. I use an ext4 partition
Code:
mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime nouser norelatime nostrictatime noiversion nobarrier noauto_da_alloc nouser_xattr data=writeback commit=30 inode_readahead_blks=64 errors=continue
Code:
# [*]nosuid - Do not allow set-user-identifier or set-group-identifier bits to take effect.
# [*]nodev - Do not interpret character or block special devices on the file system.
# [*]noatime - Do not update inode access times on this filesystem.
# [*]nodiratime - Do not update directory inode access times on this filesystem.
# [*]nouser - Forbid an ordinary (i.e., non-root) user to mount the filesystem. This is the default.
# [*]norelatime - Do not use relatime feature. See also the strictatime mount option.
# [*]nostrictatime - Use the kernel's default behaviour for inode access time updates.
# [*]noiversion - Do not increment the i_version inode field.
# [**]nobarrier - This enables/disables barriers. nobarrier disables it, barrier enables it. Write barriers enforce proper on-disk ordering of journal commits, making volatile disk write caches safe to use, at some performance penalty.
# data=writeback - Data ordering is not preserved - data may be written into the main filesystem after its metadata has been committed to the journal. This is rumoured to be the highest-throughput option. It guarantees internal filesystem integrity, however it can allow old data to appear in files after a crash and journal recovery.
# commit=30 - Sync all data and metadata every 30 seconds. The default value is 5 seconds. Zero means default. (Setting it to very large values will improve performance.)
# noauto_da_alloc - http://forum.xda-developers.com/showthread.php?t=876069
# nouser_xattr - Support "user." extended attributes (or not).
# errors=continue - errors={continue|remount-ro|panic} Define the behaviour when an error is encountered. (Either ignore errors and just mark the filesystem erroneous and continue,
# inode_readahead_blks=64 -set to 64 from default 32
thewisenerd said:
Note: memory increasing kernels and, kernels that mount certain partitions(like cache, data etc. etc. ) both do the exact same thing.
Clarifications:
1. There's no such thing as a memory increasing kernel, and there can't be. No!? Why not? Because, however much memory's present, so, remains. This particular *myth* comes from the slang "memory increasing scripts", i.e. scripts that mount an external SD Card's partition as the internal /data partition.
2. The kernel doesn't increase the memory (check clarification 1). The increasing in memory is done by reverse-mounting (yes, reverse-mounting partitions, is a fairly popular slang for this practice) partitions from the SDCard as the internal partitions, and thus, the phone thinks that it has a more storage, than its own internal storage.
So, how does the memory increasing get done?
Ramdisks! These are the files that go on form the root file system. The kernel itself is packed with the ramdisk. A typical android bootup sequence is this: (thanks to the writers of this article here: http://elinux.org/Android_Booting)
1. The first program which runs on any Android system is the bootloader. Technically, the bootloader is outside the realm of Android itself, and is used to do very low-level system initialization, before loading the Linux kernel. The kernel then does the bulk of hardware, driver and file system initialization, before starting up the user-space programs and applications that make up Android.
2. 'init'
A key component of the Android bootup sequence is the program 'init', which is a specialized program for initializing elements of the Android system. Unlike other Linux systems (embedded or otherwise), Android uses its own initialization program. (Linux desktop systems have historically used some combination of /etc/inittab and sysV init levels - e.g. /etc/rc.d/init.d with symlinks in /etc/rc.d/rc.). Some embedded Linux systems use simplified forms of these -- such as the init program included in busybox, which processes a limited form of /etc/inittab, or a direct invocation of a shell script or small program to do fixed initialization steps.
The Android 'init' program processes two files, executing the commands it finds in them, called 'init.rc' and 'init.<machine_name>.rc', where <machine_name> is the name of the hardware that Android is running on. (Usually, this is a code word. The name of the HTC1 hardware for the ADP1 is 'trout', and the name of the emulator is 'goldfish'.
The 'init.rc' file is intended to provide the generic initialization instructions, while the 'init.<machine_name>.rc' file is intended to provide the machine-specific initialization instructions.
====================================================================================================
Now, that's the general booting process. Now, lets look into our phone's booting process. The bootloader is "HBOOT" specialised for our phones, made by HTC.
This boots up the hardware, loads the kernel, and, the "init" process starts running. If you unpack any kernel you'd find these two files:
Code:
init.rc
init.pico.rc
As you'd have guessed by now, the init.rc is general instructions, and init.pico.rc is the hardware device specific parts.
The init process is what will set up all native services and this is similar to a regular Linux system boot.
So, the init process is also the reason why the filesystems are mounted. If you'd open up any init.rc file, you'd find these lines:
Code:
on fs
# mount mtd partitions
# Mount /system rw first to give the filesystem a chance to save a checkpoint
mount yaffs2 [email protected] /system
mount yaffs2 [email protected] /system ro remount
mount yaffs2 [email protected] /data nosuid nodev
mount yaffs2 [email protected] /cache nosuid nodev
This language is commonly known as the "Android Init Language", and you can look up for help here: https://android.googlesource.com/platform/system/core/+/master/init/readme.txt
So, this is where the filesystems get mounted.
Now, to make a reverse-mounting boot.img, we'd need to modify these lines. Lets mount the second partition in the SD Card in the data partition. So, we'd simply replace this line:
Code:
mount yaffs2 [email protected] /data nosuid nodev
with a modified line, like this:
Code:
mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev
If you'd see, this follows this configuration:
mount <type> <device> <dir> [ <mountoption> ]*
Now, the nosuid, nodev, etc are mountoptions. There are a variety of mount options, and the right choice of them, is likely to make your mounted partition accessible faster. Example: You can disable journaling, or use a different method of journaling, so that, you get access to the partitions faster, and as a direct result, your phone *might* become faster. The general mount options can be found here: http://linux.die.net/man/8/mount
So, that's about how you do the "reverse-mounting" thing.
p.s. here's the mount config that I usually use. Also posting the reasons
p.p.s. I use an ext4 partition
Code:
mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime nouser norelatime nostrictatime noiversion nobarrier noauto_da_alloc nouser_xattr data=writeback commit=30 inode_readahead_blks=64 errors=continue
Code:
# [*]nosuid - Do not allow set-user-identifier or set-group-identifier bits to take effect.
# [*]nodev - Do not interpret character or block special devices on the file system.
# [*]noatime - Do not update inode access times on this filesystem.
# [*]nodiratime - Do not update directory inode access times on this filesystem.
# [*]nouser - Forbid an ordinary (i.e., non-root) user to mount the filesystem. This is the default.
# [*]norelatime - Do not use relatime feature. See also the strictatime mount option.
# [*]nostrictatime - Use the kernel's default behaviour for inode access time updates.
# [*]noiversion - Do not increment the i_version inode field.
# [**]nobarrier - This enables/disables barriers. nobarrier disables it, barrier enables it. Write barriers enforce proper on-disk ordering of journal commits, making volatile disk write caches safe to use, at some performance penalty.
# data=writeback - Data ordering is not preserved - data may be written into the main filesystem after its metadata has been committed to the journal. This is rumoured to be the highest-throughput option. It guarantees internal filesystem integrity, however it can allow old data to appear in files after a crash and journal recovery.
# commit=30 - Sync all data and metadata every 30 seconds. The default value is 5 seconds. Zero means default. (Setting it to very large values will improve performance.)
# noauto_da_alloc - http://forum.xda-developers.com/showthread.php?t=876069
# nouser_xattr - Support "user." extended attributes (or not).
# errors=continue - errors={continue|remount-ro|panic} Define the behaviour when an error is encountered. (Either ignore errors and just mark the filesystem erroneous and continue,
# inode_readahead_blks=64 -set to 64 from default 32
Click to expand...
Click to collapse
Three words - YOU ARE GREAT! I mean, in the whole explanation you did, I just knew that there is some reverse mounting in init.rc in which few partitions are mounted as data/cache and you cleared my concepts. One more thing related to this I'd like to ask, in sense based roms(For our device) is it possible to do changes in init.rc(which I guess would be required) so that apps are directly installed in sd-ext and the symlink also gets done.
Once again,
THANKS
thewisenerd said:
Note: memory increasing kernels and, kernels that mount certain partitions(like cache, data etc. etc. ) both do the exact same thing.
Clarifications:
Click to expand...
Click to collapse
Lol thats a huge explanation! :good: :highfive:
#Superuser said:
Three words - YOU ARE GREAT! I mean, in the whole explanation you did, I just knew that there is some reverse mounting in init.rc in which few partitions are mounted as data/cache and you cleared my concepts. One more thing related to this I'd like to ask, in sense based roms(For our device) is it possible to do changes in init.rc(which I guess would be required) so that apps are directly installed in sd-ext and the symlink also gets done.
Once again,
THANKS
Click to expand...
Click to collapse
Yes, it is possible. Open your init.rc and find the following line:
Code:
mkdir /data/app 0771 system system
mkdir /data/app-private 0771 system system
mkdir /data/app-asec 0700 root root
mkdir /data/app-lib 0771 system system
Now change it like this:
Code:
# For installing apps directly to /sd-ext:
mkdir /sd-ext/app 0771 system system
mkdir /sd-ext/app-private 0771 system system
mkdir /sd-ext/app-asec 0700 root root
mkdir /sd-ext/app-lib 0771 system system
# Now the symlinking part:
symlink /sd-ext/app /data/app
symlink /sd-ext/app-private /data/app-private
symlink /sd-ext/app-asec /data/app-asec
symlink /sd-ext/app-lib /data/app-lib
NOTE: Before doing this make sure that you have mounted /sd-ext partition.
@thewisenerd....excellent post.....cleared so many doubts....You are really great.
I have a question, I am no dev but just trying to learn some basics about android. As u said that ramdisk does the job of mounting sd partitions using the reverse mounting thing. What is the job of the scripts like int2ext or ungaze or mount2sd.
Is it like when the reverse mounting is not done in ramdisk we need to use these scripts. If yes, then how do these scripts talk 2 ramdisk or kernel to tell them to mount these sd partitions.
I know this may sound noob to you. But m just trying to learn some basics.
@cute_prince Thanks. Now, I'm gonna post all my doubt related to kernels and ramdisks! Thanks to @thewisenerd as well!
cuteitsme said:
@thewisenerd....excellent post.....cleared so many doubts....You are really great.
I have a question, I am no dev but just trying to learn some basics about android. As u said that ramdisk does the job of mounting sd partitions using the reverse mounting thing. What is the job of the scripts like int2ext or ungaze or mount2sd.
Is it like when the reverse mounting is not done in ramdisk we need to use these scripts. If yes, then how do these scripts talk 2 ramdisk or kernel to tell them to mount these sd partitions.
I know this may sound noob to you. But m just trying to learn some basics.
Click to expand...
Click to collapse
Everyone's a newbie (unless they remain to stay a n00b). Anyways, let's get back to on-topic.
So... How does these files in
Code:
/system/init.d/<insert-script-name>
cause the reverse-mount?
For this, we look back, into the init.rc and init.pico.rc files
If you'd read this, https://android.googlesource.com/platform/system/core/+/master/init/readme.txt, you'd find that the Android Init Language gives you an "exec" command for running apps/scripts. So, you should have gotten the hint by now
So, open up init.rc and search for any "exec" command. I can just make it easier, but I want you to find any "exec" call that runs processes from /system/*.
OFC, I'd be telling the answer below, but I want you to find it too
So, here's the key. You're likely to find lines as the same as, or similar to below lines:
Code:
# Run sysinit
exec /system/bin/sysinit
in any one of the *.rc files (mostly init.rc or init.pico.rc).
So, lets take a look at this file.
Below is the above file (/system/bin/sysinit) from CM10.2, weekly 9
Code:
#!/system/bin/sh
export PATH=/sbin:/system/sbin:/system/bin:/system/xbin
/system/bin/logwrapper /system/xbin/run-parts /system/etc/init.d
So, what's this, exactly?
Linux users would be familiar with the PATH variable name
And, logwrapper? Here's standard help:
Code:
Usage: logwrapper [-d] BINARY [ARGS ...]
Forks and executes BINARY ARGS, redirecting stdout and stderr to
the Android logging system. Tag is set to BINARY, priority is
always LOG_INFO.
-d: Causes logwrapper to SIGSEGV when BINARY terminates
fault address is set to the status of wait()
So, its going to execute a binary file, but which?
The next "argument" reads "/system/xbin/run-parts"
Again, here's standard help:
Code:
BusyBox v1.20.2-cm9 bionic (2012-11-18 13:31 +0100) multi-call binary.
Usage: run-parts [-t] [-l] [-a ARG] [-u MASK] DIRECTORY
Run a bunch of scripts in DIRECTORY
-t Print what would be run, but don't actually run anything
-a ARG Pass ARG as argument for every program
-u MASK Set the umask to MASK before running every program
-l Print names of all matching files even if they are not executable
So, run a bunch of scripts in a directory?
That pretty much explains why the next "argument" follows as "/system/etc/init.d ".
So, that's how init.d works :cyclops:
============================================================================================================
Now, moving on to memory increasing scripts (oh! I hate that slang)
Anyways, so, we found out that during the boot, the init.rc file is calling the /system/bin/sysinit file. If you'd notice, a few lines "above",
Here's something from http://www.kpbird.com/2012/11/in-depth-android-boot-sequence-process.html
Below is the sequence of android booting. Note that until all these processes are completed, you still see the boot logo. So, if your phone's struck at green htc screen, then, any one of this processes is.. hung.
on early-init:
Set init and its forked children's oom_adj.
Set the security context for the init process.
on init
setup the global environment
Create cgroup mount point for cpu accounting and many other things...
on fs
mount mtd partitions
on post-fs
change permissions of system directories
on post-fs-data
change permission of /data folders and sub folders
on boot
basic network init ,Memory Management ,etc
service servicemanager
start system manager to manage all native services like location, audio, shared preference etc..
service zygote
start zygote as app_process
So, the "/system/bin/sysinit" runs at the "boot" service. Also, it is run by the bootloader. So, it has full access to the root file system that the ramdisk creates. Now, to be noted: the "boot" service runs after the "fs" service, evidently, because else, you wouldn't be able to access the "/system/bin/sysinit" otherwise.
So, the "sysinit" script runs, running all the files from /system/etc/init.d with the help of busybox
That's about it
But, how does the reverse-mounting take place!?
Let me take the example of the simplest reverse mounting script I've ever found: int2ext. I've seen the mounts2sd script, and think its bloated, IMO, because why have a 1000+ lines script, when a script with >40 lines can do it :angel: ) (no offense). For me, just placing script, setting permissions, rebooting should increase the memory of my device. No roundabout stuff.
So, I'd be explaining how int2ext works, below:
Note: before you proceed, you'd have noticed that the sysinit file set the PATH variable. Its the location for all the accessible binary files. (if you don't understand this, using linux might help you understand this better ) This is similar to the PATH variable in windows too.. This just tells the system where to look for executable binary files/programs.
So, here's a very minimal int2ext script (modified to make this post smaller):
The purple lines ("Royal Blue" according to XDA), are the lines of code. Rest are my comments.
Credits go to original file, got from here: https://github.com/croniccorey/cronmod-scripts/blob/master/int2ext%20scripts/INT2EXT
## Only continue if mmcblk0p2 exists
if [ ! -e /dev/block/mmcblk0p2 ] //If /dev/block/mmcblk0p2 doesn't exist
then //then
exit //this script exists. this code's here for safety.
fi;
## Set SD cache size
SD_CACHE=/sys/devices/virtual/bdi/179:0/read_ahead_kb //read_ahead_kb is the amount of kb that the kernel reads, beforehand.
if [ -e $SD_CACHE ] //checking the existence of the file, just in case...
then
busybox echo "2048" > $SD_CACHE; //2048 is found to be an optimum value for sdcards, class 4 and better.
fi;
## Make /sd-ext directory if needed and unmount /sd-ext if it already mounted
##why? because the writer croniccorey, has done some thinking here
##will explain side by side... If the directory /sd-ext doesn't exist, its created here.
##note: the commands run by the init.rc have full access to the root file system created.
if [ ! -e /sd-ext ] //if doesn't exist sd-ext foler in root file system
then
busybox mount -o remount,rw /; //mount the root file system "/" as rewritable
busybox mkdir /sd-ext; //create a directory named sd-ext
busybox mount -o remount,ro /; //mount the file system as read-only again
else
busybox umount /sd-ext; //else, unmount the sd-ext partition, i.e. partition in the sd-card.
fi;
## Move /data mount point to /sd-ext
INT_DATA=$(busybox mountpoint -n /data | cut -d ' ' -f1) //this gets the mountpoint of the data partition.
// this command "busybox mountpoint -n /data | cut -d ' ' -f1" is actually runnable, and you'd get the mount point
// of the /data partition in the internal memory (which is this: /dev/block/mtdblock5)
busybox umount /data; //unmount the /data partition too/
// the data partition is unmounted because we're going to do some stuff
busybox mount $INT_DATA /sd-ext;
// mount the internal data partition in the sd-ext folder
## Mount mmcblk0p2 to /data
busybox mount -o noatime,nodiratime,nosuid,nodev /dev/block/mmcblk0p2 /data; //well, this is self-explanatory. mount the partition in sdcard into internal memory.
busybox chown 1000:1000 /data; //chown the folder by root (get more help by running "man chown" in a shell)
busybox chmod 771 /data; //chmod the folder (get more help by running "man chmod" in a shell)
## Move existing files
if [ ! -e /data/app ] //why /data/app? simply, it can also be anything else like /data/data...
then
busybox mv /sd-ext/* /data;
// the sd-ext folder has the files of the internal /data partition
// those are moved to the sd-card's partition, which is mounted in /data now.
fi;
## Unmount /sd-ext
//unmount the internal data partition.
// we have the partition from the sd-card mounted in the /data partition currently.
busybox umount /sd-ext;
sync;
//sync changes with file system
Click to expand...
Click to collapse
the comments should explain almost everything...
This script, is almost flawless. Couldn't find any bugs in it. Does what would have been done by changing the mount point in the "on fs" part, where it mounts the file system.
An added advantage is that this can be put into use any time you want... Example, you use your phone, internal memory gets filled up, just put this script in /system/etc/init.d, and set permissions, and reboot! voila! memory increased!
Note: reversing this can't be done by just deleting the script. an appropriate script that moves back user data to internal partitions may be needed.
Also note:
You can always tweak this command:
busybox mount -o noatime,nodiratime,nosuid,nodev /dev/block/mmcblk0p2 /data;
for better performance. Check my config, in post 8 or so... 1'st page..
You should've also guessed why we set permissions to executable, by now... Else, the file wouldn't be able to "execute"
P.S. Adding a few lines to init.rc should make it possible to have no permission change, but, that can wait for another day...
Simply awesome.....what an explanation sirji....
Still trying to understand some part but i will have to do more reading for that first....that i will do.....but must say U rock man.....thanks a lot for this....
thewisenerd said:
Everyone's a newbie (unless they remain to stay a n00b). Anyways, let's get back to on-topic.
So... How does these files in
Code:
/system/init.d/<insert-script-name>
cause the reverse-mount?
For this, we look back, into the init.rc and init.pico.rc files
If you'd read this, https://android.googlesource.com/platform/system/core/+/master/init/readme.txt, you'd find that the Android Init Language gives you an "exec" command for running apps/scripts. So, you should have gotten the hint by now
So, open up init.rc and search for any "exec" command. I can just make it easier, but I want you to find any "exec" call that runs processes from /system/*.
OFC, I'd be telling the answer below, but I want you to find it too
So, here's the key. You're likely to find lines as the same as, or similar to below lines:
Code:
# Run sysinit
exec /system/bin/sysinit
in any one of the *.rc files (mostly init.rc or init.pico.rc).
So, lets take a look at this file.
Below is the above file (/system/bin/sysinit) from CM10.2, weekly 9
Code:
#!/system/bin/sh
export PATH=/sbin:/system/sbin:/system/bin:/system/xbin
/system/bin/logwrapper /system/xbin/run-parts /system/etc/init.d
So, what's this, exactly?
Linux users would be familiar with the PATH variable name
And, logwrapper? Here's standard help:
Code:
Usage: logwrapper [-d] BINARY [ARGS ...]
Forks and executes BINARY ARGS, redirecting stdout and stderr to
the Android logging system. Tag is set to BINARY, priority is
always LOG_INFO.
-d: Causes logwrapper to SIGSEGV when BINARY terminates
fault address is set to the status of wait()
So, its going to execute a binary file, but which?
The next "argument" reads "/system/xbin/run-parts"
Again, here's standard help:
Code:
BusyBox v1.20.2-cm9 bionic (2012-11-18 13:31 +0100) multi-call binary.
Usage: run-parts [-t] [-l] [-a ARG] [-u MASK] DIRECTORY
Run a bunch of scripts in DIRECTORY
-t Print what would be run, but don't actually run anything
-a ARG Pass ARG as argument for every program
-u MASK Set the umask to MASK before running every program
-l Print names of all matching files even if they are not executable
So, run a bunch of scripts in a directory?
That pretty much explains why the next "argument" follows as "/system/etc/init.d ".
So, that's how init.d works :cyclops:
============================================================================================================
Now, moving on to memory increasing scripts (oh! I hate that slang)
Anyways, so, we found out that during the boot, the init.rc file is calling the /system/bin/sysinit file. If you'd notice, a few lines "above",
Here's something from http://www.kpbird.com/2012/11/in-depth-android-boot-sequence-process.html
Below is the sequence of android booting. Note that until all these processes are completed, you still see the boot logo. So, if your phone's struck at green htc screen, then, any one of this processes is.. hung.
on early-init:
Set init and its forked children's oom_adj.
Set the security context for the init process.
on init
setup the global environment
Create cgroup mount point for cpu accounting and many other things...
on fs
mount mtd partitions
on post-fs
change permissions of system directories
on post-fs-data
change permission of /data folders and sub folders
on boot
basic network init ,Memory Management ,etc
service servicemanager
start system manager to manage all native services like location, audio, shared preference etc..
service zygote
start zygote as app_process
So, the "/system/bin/sysinit" runs at the "boot" service. Also, it is run by the bootloader. So, it has full access to the root file system that the ramdisk creates. Now, to be noted: the "boot" service runs after the "fs" service, evidently, because else, you wouldn't be able to access the "/system/bin/sysinit" otherwise.
So, the "sysinit" script runs, running all the files from /system/etc/init.d with the help of busybox
That's about it
But, how does the reverse-mounting take place!?
Let me take the example of the simplest reverse mounting script I've ever found: int2ext. I've seen the mounts2sd script, and think its bloated, IMO, because why have a 1000+ lines script, when a script with >40 lines can do it :angel: ) (no offense). For me, just placing script, setting permissions, rebooting should increase the memory of my device. No roundabout stuff.
So, I'd be explaining how int2ext works, below:
Note: before you proceed, you'd have noticed that the sysinit file set the PATH variable. Its the location for all the accessible binary files. (if you don't understand this, using linux might help you understand this better ) This is similar to the PATH variable in windows too.. This just tells the system where to look for executable binary files/programs.
So, here's a very minimal int2ext script (modified to make this post smaller):
The purple lines ("Royal Blue" according to XDA), are the lines of code. Rest are my comments.
Credits go to original file, got from here: https://github.com/croniccorey/cronmod-scripts/blob/master/int2ext%20scripts/INT2EXT
the comments should explain almost everything...
This script, is almost flawless. Couldn't find any bugs in it. Does what would have been done by changing the mount point in the "on fs" part, where it mounts the file system.
An added advantage is that this can be put into use any time you want... Example, you use your phone, internal memory gets filled up, just put this script in /system/etc/init.d, and set permissions, and reboot! voila! memory increased!
Note: reversing this can't be done by just deleting the script. an appropriate script that moves back user data to internal partitions may be needed.
Also note:
You can always tweak this command:
busybox mount -o noatime,nodiratime,nosuid,nodev /dev/block/mmcblk0p2 /data;
for better performance. Check my config, in post 8 or so... 1'st page..
You should've also guessed why we set permissions to executable, by now... Else, the file wouldn't be able to "execute"
P.S. Adding a few lines to init.rc should make it possible to have no permission change, but, that can wait for another day...
Click to expand...
Click to collapse
I read somewhere, that init.d scripts run alphabetically, maybe this is the reason int2ext is named as 40int2ext so that it starts first?
#Superuser said:
I read somewhere, that init.d scripts run alphabetically, maybe this is the reason int2ext is named as 40int2ext so that it starts first?
Click to expand...
Click to collapse
Yeah....I guess that's the linux thing. I think the higher the no. the late the script will be called. This is basically to run the more important scripts before the other scripts.
hi i unpacked the boot.img files of cm11 beta 1 and 2 in order to understand this better and understood many things which u have explained.
I did found those lines starting with exec in some init. files.
Thanks a lot for this. Since u must be aware that the sd-ext is not mounting in buid 2 and thus int2ext is not working. So, out of curiosity I was trying to understand the difference between these two files from different builds(beta 1 and 2) but not able to find any.
Can u give any hint as to why even after having the same lines in fstab.pico of build 1 and build 2 int2ext was working in buld 1 and not in build 2. Of course by doing reverse mounting we can overcome this issue. But I am just trying to understand the difference between the two builds at ramdisk level.
cuteitsme said:
hi i unpacked the boot.img files of cm11 beta 1 and 2 in order to understand this better and understood many things which u have explained.
I did found those lines starting with exec in some init. files.
Thanks a lot for this. Since u must be aware that the sd-ext is not mounting in buid 2 and thus int2ext is not working. So, out of curiosity I was trying to understand the difference between these two files from different builds(beta 1 and 2) but not able to find any.
Can u give any hint as to why even after having the same lines in fstab.pico of build 1 and build 2 int2ext was working in buld 1 and not in build 2. Of course by doing reverse mounting we can overcome this issue. But I am just trying to understand the difference between the two builds at ramdisk level.
Click to expand...
Click to collapse
Hmm... Could you post the two boot.img's here? I'd like to have a look
thewisenerd said:
Hmm... Could you post the two boot.img's here? I'd like to have a look
Click to expand...
Click to collapse
Sure y not.....here u go
@thewisenerd: does the ION Kernel used in cyanogenmod 11 support swap
PiCo HackR said:
@thewisenerd: does the ION Kernel used in cyanogenmod 11 support swap
Click to expand...
Click to collapse
The question is: Why do you need swap?

Nexus 5 32GB shows only 12.55GB. Cannot Use Stock Recovery, gets stuck on formating.

I'm having the issue of missing storage with a 32GB phone.
Trying the instructions here - https://forum.xda-developers.com/showpost.php?p=47156078&postcount=2
The issue is the stock recovery doesn't seem to work for me. When you go to recovery mode with it flashed it gets stuck on the android symbol standing up with the spinning gear. I've left this overnight for over 10 hours.
I can resolve this if I flash TWRP recovery which then enables the phone to boot into android setup properly but it then reverts back to the issue of missing storage.
32 become 16 GB, the last last last attempt to solve the issue
Quote:
Originally Posted by GioVIP
i did just like you said but got again 16 instead of 32
Click to expand...
Click to collapse
I was in the same situation, metod 1, metod 2 any method to fix it did not work
I've solved it by doing this (make this at your own risk, FIRST be sure that your emmc is not damaged):
root your device, and put parted binary ( find it in the forum) in system/bin with execution permission.
Then reboot in twrp recovery, then in twrp recovery terminal or in adb root shell, type this :
Code:
Code:
mount system
umount /dev/block/mmcblk0p28
parted /dev/block/mmcblk0
check 28
it should say something like
Code:
Code:
Error: The backup GPT table is not at the end of the disk, as it should be.
This might mean that another operating system believes the disk is smaller.
Fix, by moving the backup to the end (and removing the old backup)?
Fix/Ignore/Cancel? Fix
Warning: Not all of the space available to /dev/block/mmcblk0 appears to be
used, you can fix the GPT to use all of the space (an extra xxxxxx blocks) or
continue with the current setting?
Fix/Ignore?Fix
answer Fix and Fix.
now type:
Code:
Code:
print
the last two partitions should be:
Code:
28 2032MB 15.8GB 13.7GB userdata
29 15.8GB 15.8GB 5632B grow
if EVERYTHING is like that, let's make a try, type:
Code:
Code:
mkpart primary ext4 15.8GB 25.8GB name 30 trypart
name 30 trypart
print
if you can make trypart without errors and you can see trypart with the print command then
you can proceed to the next step, that will ERASE your data ( worse your userdata partition, before recreating it), type:
Code:
Code:
rm 30
rm 29
rm 28
mkpart primary ext4 2032140288B 31268511743B name 28 userdata
name 28 userdata
mkpart primary 31268511744B 31268517375B name 29 grow
name 29 grow
print
you should see the userdata partition with the new size, and the grow partition after.
Exit from parted by typing
Code:
Code:
quit
Now reboot in fastboot mode and from pc terminal type:
Code:
Code:
fastboot format userdata
ok reboot (wait it take its time) it should be solved.
16 to 32
1.Flash stock rom
2.Wait for opening
3.Reset ur nexus 5 (using options in setting app)
4.*important* Reset when phone boot completely..

Patching Adbd Binary

Hi everyone, I'm trying to patch the adbd binary in my phone (rooted and with Android 11) in order to run it as root.
I found out that the binary was located in /apex/com.android.adbd/bin/, which, according to this link, comes from /system/apex/com.android.adbd.apex.
The apex file can be opened like a zip, so I extracted the files and found the apex_payload.img file, which is indeed mounted in /apex/com.android.adbd/.
Now, the problem is that I have to mount the img file in order to patch it, so I tried to do it with the command "mount -t ext4 -o loop apex_payload.img apex_payload" (yes, this folder exists) but I got the error "mount: '/dev/block/loop28'->'apex_payload': Invalid argument", and if I repeat the command the number after loop increases by one every time. I thought that maybe the image file had a read-only lock, like many partitions such as system,vendor,product etc..., and in fact the image has the shared_blocks feature (checked with "tune2fs -l apex_payload.img"), and I tried to remove it with "e2fsck -y -E unshare_blocks apex_payload.img", and this is the output: e2fsck 1.45.4 (23-Sep-2019)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information
apex_payload.img: 38/48 files (2.6% non-contiguous), 1932/1939 blocks
It doesn't return errors, and the exit code is 0, but it doesn't actually remove the read-only lock on the image file (checked again with tune2fs), so now I'm stuck with that problem. Does anyone know how to solve this or if there are other ways to modify an ext4 image?
Thank you very much.
hmmm. im trying to do something similar, but i have a system shell -- you have any luck?

Categories

Resources