Related
Just a quick thought - I was trying to figure out if I might get somewhat better filesystem performance out of ext4 rather than ext3 for the user data partition. As a first pass, I compiled a kernel with ext4 enabled, and tried to change my boot partition init.rc to mount /data as an ext4 partition rather than an ext3 - my understanding is they should be mount-compatible even without repartitioning.
The result of this was a bootloop. Had to reflash back to a working boot.img. Didn't do anything bad to my /data partition though.
Any ideas why this might be happening? Is there a way to grab dmesg from a boot looping kernel on Android?
I've found the following procedures to be helpful in debugging custom kernels.
First, remove the fbcon.c patch in the kernel source (you can just edit the one line that gets modified and uncomment it):
Code:
nano drivers/video/console/fbcon.c
Search for "navy", then uncomment the line:
Code:
update_screen(vc);
Rebuild your kernel. Now you'll have the ability to get a framebuffer console, if your kernel command line arguments are correct. You have to supply custom kernel command line arguments when using mk-boot-img. I use the following when using mk-boot-image:
Code:
mkbootimg --kernel zImage --ramdisk initrd.gz --cmdline "[email protected] [email protected] vmalloc=192M video=tegrafb console=tty0,115200 fbcon=rotate:1 fbcon=font:VGA8x8 usbcore.old_scheme_first=1 cpuid=200102 devicetype=1002 btmac=9c5ed6131a00 tegraboot=nand mtdparts=tegra_nand:[email protected](misc),[email protected](recovery),[email protected](boot),[email protected](system),[email protected](cache),[email protected](bootbmp),[email protected](logodata)" -o boot.img
The important part there is changing the console parameter from the default ttyS0 to tty0.
I make separate boot images when debugging - one that executes a shell in the initial ramdisk, and one that doesn't. To get a shell in the initrd, which you'll want to do in this case to figure out your mount issue, is change the above mkbootimg --cmdline parameter:
Code:
mkbootimg --kernel zImage --ramdisk initrd.gz --cmdline "break=top [email protected] [email protected] vmalloc=192M video=tegrafb console=tty0,115200 fbcon=rotate:1 fbcon=font:VGA8x8 usbcore.old_scheme_first=1 cpuid=200102 devicetype=1002 btmac=9c5ed6131a00 tegraboot=nand mtdparts=tegra_nand:[email protected](misc),[email protected](recovery),[email protected](boot),[email protected](system),[email protected](cache),[email protected](bootbmp),[email protected](logodata)" -o boot.img
Notice the "break=top". This will give you a command shell before anything in initrc has been executed. You could also use "break=bottom" or "break=init" to get you to different spots in initrc.
Hope that helps! Happy hacking!
Oh, and I forgot to mention that a usb keyboard is very handy after you get a shell
jersacct said:
I've found the following procedures to be helpful in debugging custom kernels.
First, remove the fbcon.c patch in the kernel source (you can just edit the one line that gets modified and uncomment it):
Code:
nano drivers/video/console/fbcon.c
Search for "navy", then uncomment the line:
Code:
update_screen(vc);
Rebuild your kernel. Now you'll have the ability to get a framebuffer console, if your kernel command line arguments are correct. You have to supply custom kernel command line arguments when using mk-boot-img. I use the following when using mk-boot-image:
Code:
mkbootimg --kernel zImage --ramdisk initrd.gz --cmdline "[email protected] [email protected] vmalloc=192M video=tegrafb console=tty0,115200 fbcon=rotate:1 fbcon=font:VGA8x8 usbcore.old_scheme_first=1 cpuid=200102 devicetype=1002 btmac=9c5ed6131a00 tegraboot=nand mtdparts=tegra_nand:[email protected](misc),[email protected](recovery),[email protected](boot),[email protected](system),153728[email protected](cache),[email protected](bootbmp),[email protected](logodata)" -o boot.img
The important part there is changing the console parameter from the default ttyS0 to tty0.
I make separate boot images when debugging - one that executes a shell in the initial ramdisk, and one that doesn't. To get a shell in the initrd, which you'll want to do in this case to figure out your mount issue, is change the above mkbootimg --cmdline parameter:
Code:
mkbootimg --kernel zImage --ramdisk initrd.gz --cmdline "break=top [email protected] [email protected] vmalloc=192M video=tegrafb console=tty0,115200 fbcon=rotate:1 fbcon=font:VGA8x8 usbcore.old_scheme_first=1 cpuid=200102 devicetype=1002 btmac=9c5ed6131a00 tegraboot=nand mtdparts=tegra_nand:[email protected](misc),[email protected](recovery),[email protected](boot),[email protected](system),[email protected](cache),[email protected](bootbmp),[email protected](logodata)" -o boot.img
Notice the "break=top". This will give you a command shell before anything in initrc has been executed. You could also use "break=bottom" or "break=init" to get you to different spots in initrc.
Hope that helps! Happy hacking!
Click to expand...
Click to collapse
Hey jersacct, so I've tried this (recompiled with your suggested patch, then made new boot image with the last set of params you mentioned) and I absolutely see the framebuffer scrolling by rapidly right after the Viewsonic Birds display. However, I never get the break into console - the FB scrolls by very quickly, then I get the GTabDevs boot image for about 2 seconds, then I boot loop again.
Any ideas? I have tried both the break=top and break=init variants of that mkbootimg command line with no success.
EDIT: I tried with a known-good ramdisk image too. The boot.img boots fine, but again I never get dropped to console.
Hmm, I suspected the issue might be that the default config has a pesky setting called CONFIG_CMDLINE="" that could be force-overriding any command line passed to the kernel from the boot image. Tried again with that line commented out, and still didn't have any luck getting the break= command to do anything.
I could probably try forcing the command line from CONFIG_CMDLINE but not sure why that would be different from passing it in mkbootimg...
EDIT: nope, I tried that too. Doesn't make a difference. So I assume it's receiving the CMDLINE just fine. For some reason break=top isn't giving me a console. I also tried it with my USB keyboard already plugged in to see if that was making a difference with the input devices, and still no console.
Still no luck on the debug console. At my wit's end on that. All I can say is that on a regular boot with ext3 mounting my /data partition I see that the partition is dirty and the first mount attempt fails. Second attempt seems to succeed. Says I need to fsck my data partition - which I'd do, but our busybox doesn't seem to support fsck.
Anybody know where I can find a busybox binary that properly supports fsck for ext2/3 and also has mkfs.ext4? I know the Archos guys seem to have one and that's an ARM v9 device so it should work, but I'm having trouble finding it (I even downloaded the "SDE" firmware from the Archos site).
I think the reason the ext4 driver is probably just that it's dirty, though I can't confirm that without the damned debug console.
If I can get an up-to-date busybox on here, I suspect this will just start working magically, without further ado. Ideas are appreciated, either on busybox binaries or on getting the initramdisk console stuff that jersacct posted to work.
These guys can help you:
http://forum.xda-developers.com/showthread.php?t=895599
And it makes a HUUUUGE difference...
rcgabriel said:
Still no luck on the debug console. At my wit's end on that. All I can say is that on a regular boot with ext3 mounting my /data partition I see that the partition is dirty and the first mount attempt fails. Second attempt seems to succeed. Says I need to fsck my data partition - which I'd do, but our busybox doesn't seem to support fsck.
Anybody know where I can find a busybox binary that properly supports fsck for ext2/3 and also has mkfs.ext4? I know the Archos guys seem to have one and that's an ARM v9 device so it should work, but I'm having trouble finding it (I even downloaded the "SDE" firmware from the Archos site).
I think the reason the ext4 driver is probably just that it's dirty, though I can't confirm that without the damned debug console.
If I can get an up-to-date busybox on here, I suspect this will just start working magically, without further ado. Ideas are appreciated, either on busybox binaries or on getting the initramdisk console stuff that jersacct posted to work.
Click to expand...
Click to collapse
You know, I didn't think to mention that I was using the Karmic ramdisk image - I bet our stock ramdisk image doesn't have breakpoints setup in initrc. You can grab a copy of a stock initrd image here:
http://www.retardedrobot.com/karmic-initrd-orig.gz
Then just use it instead of the stock ramdisk image when using mkbootimg. Please note, I hardcoded mine to get root to work off the SD card. I think this is an unmodified initrd, so it may work straight out of the box (and boot android if allowed). You might have problems with it though, and may need to unpack it, edit some scripts (for mounting root, etc) and repackage it.
Hope that helps.
stanglx said:
These guys can help you:
http://forum.xda-developers.com/showthread.php?t=895599
And it makes a HUUUUGE difference...
Click to expand...
Click to collapse
Yeah, I've seen the thread before.
I think I was just exhausted and bleary eyed earlier and completely frustrated.
Now that I'm a little more clear-headed, things are much easier.
I've snagged the busybox build out of the Archos initramfs. It indeed does all the ext4 stuff.
jersacct said:
You know, I didn't think to mention that I was using the Karmic ramdisk image - I bet our stock ramdisk image doesn't have breakpoints setup in initrc. You can grab a copy of a stock initrd image here:
http://www.retardedrobot.com/karmic-initrd-orig.gz
Then just use it instead of the stock ramdisk image when using mkbootimg. Please note, I hardcoded mine to get root to work off the SD card. I think this is an unmodified initrd, so it may work straight out of the box (and boot android if allowed). You might have problems with it though, and may need to unpack it, edit some scripts (for mounting root, etc) and repackage it.
Hope that helps.
Click to expand...
Click to collapse
Ahh, this explains why it doesn't work. Thanks so much, I'll take a crack, but hopefully I can clean up my /data partition into a proper ext4 partition with busybox and get it mounting now. I'll see tomorrow, too tired tonight.
Ba...ZING! Finally got ext4 driver working properly. Figured out what was causing the bootloops before, I think - or at least localized the issue.
Running an e2fsck and disabling inandop.sh seems to have done the trick. I think one of the checks in inandop.sh was triggering a reboot. My init.rd now successfully mounts /data as an ext4 partition, mount reports the partition as ext4.
What I need to do now is try to clean up inandop.sh so it properly uses the right binaries to set up the partition as a true ext4. Currently just mounting an ext3 partition with the ext4 driver.
So far no significant performance boost in Quadrant, it's within 50 points of where it was before. But I'm not done yet.
rcgabriel said:
Ba...ZING! Finally got ext4 driver working properly. Figured out what was causing the bootloops before, I think - or at least localized the issue.
Running an e2fsck and disabling inandop.sh seems to have done the trick. I think one of the checks in inandop.sh was triggering a reboot. My init.rd now successfully mounts /data as an ext4 partition, mount reports the partition as ext4.
What I need to do now is try to clean up inandop.sh so it properly uses the right binaries to set up the partition as a true ext4. Currently just mounting an ext3 partition with the ext4 driver.
So far no significant performance boost in Quadrant, it's within 50 points of where it was before. But I'm not done yet.
Click to expand...
Click to collapse
will it incorporate in the future Vegan release? I know we have to do complete format, and goto ext4?
Will love if we can get ext4.
Are you using internal or external (sd card) for the test? The internel storage is very very slow compared to most class 4 sd cards.
rcgabriel said:
Ba...ZING! Finally got ext4 driver working properly. Figured out what was causing the bootloops before, I think - or at least localized the issue.
Running an e2fsck and disabling inandop.sh seems to have done the trick. I think one of the checks in inandop.sh was triggering a reboot. My init.rd now successfully mounts /data as an ext4 partition, mount reports the partition as ext4.
What I need to do now is try to clean up inandop.sh so it properly uses the right binaries to set up the partition as a true ext4. Currently just mounting an ext3 partition with the ext4 driver.
So far no significant performance boost in Quadrant, it's within 50 points of where it was before. But I'm not done yet.
Click to expand...
Click to collapse
One would think that Viewsonic will atleast put class 4.. It does appear to be slow at times when you use SDCard for cache to browse..
stanglx said:
Are you using internal or external (sd card) for the test? The internel storage is very very slow compared to most class 4 sd cards.
Click to expand...
Click to collapse
Using internal SD card. I don't have an extra microSD handy here, just the one in my Nexus One, which is 16GB class 2 so probably too slow to be useful for that.
Still I'm fairly convinced that I should be able to get better results with the internal SD card.
Take a look.. See if your getting the same results with ext4..
Here is the output of a test:
http://forum.xda-developers.com/showpost.php?p=10239762&postcount=2
As you can see a Class
rcgabriel said:
Using internal SD card. I don't have an extra microSD handy here, just the one in my Nexus One, which is 16GB class 2 so probably too slow to be useful for that.
Still I'm fairly convinced that I should be able to get better results with the internal SD card.
Click to expand...
Click to collapse
stanglx said:
Take a look.. See if your getting the same results with ext4..
Here is the output of a test:
http://forum.xda-developers.com/showpost.php?p=10239762&postcount=2
As you can see a Class
Click to expand...
Click to collapse
I'll do more testing later, but right now I've managed to bootloop my G Tablet and I don't have my USB cable here, it's at the office (snow day today). I can boot to recovery but it seems like ClockworkMod recovery 0.8 doesn't want to mount my external MicroSD card from my N1 to copy over a working update.zip to re-flash.
So unless I find another USB cable lying around the apartment somewhere, my G Tablet is out of commission until Monday or until I hit the nearest Radio Shack this weekend.
Status update: I am waiting on a class 6 SD card from Newegg so I can try more stuff out and perfect this without continually screwing up my internal SD card and having to repartition, reformat, etc.
In the meantime, I have tried adapting Chenglu's data2loop method to use an ext4 filesystem in the loop device files rather than an ext2 filesystem as he uses, using a kernel with ext4 enabled.
The good - I'm getting even higher Quadrant benchmarks than with the baseline data2loop. Up to around 3470, averaging around 3400 with a couple of runs. I was, with the same kernel and Chenglu's data2loop patch, getting in the 3200s to 3300s, so the difference seemed somewhat significant to me.
Unfortunately, it seems to puke out every time I reboot. The data must get screwed up somehow in the /data/ext4 files, and my G Tab goes schizo and won't even finish the bootup process - screen flickers from the "Vegan" logo during boot, and then screen shuts off. Have to hard shut-down then hard reboot to recovery and wipe data to get things working again.
The /misc/inandop.log file seems to report that data-to-loop was successfully enabled, so I'm not sure exactly why it's puking out in such epic fashion. Have to work more on this later.
Most likely experiencing some corruption in the cache.... add the sync command to the mount and see if that helps stability using the loopback device.... Also when you create the file system are you using
mkfs.ext4 -O ^huge_file ....
I have read you need to turn off huge_file as it causes some issues (dont remember where I read it though)
rcgabriel said:
Status update: I am waiting on a class 6 SD card from Newegg so I can try more stuff out and perfect this without continually screwing up my internal SD card and having to repartition, reformat, etc.
In the meantime, I have tried adapting Chenglu's data2loop method to use an ext4 filesystem in the loop device files rather than an ext2 filesystem as he uses, using a kernel with ext4 enabled.
The good - I'm getting even higher Quadrant benchmarks than with the baseline data2loop. Up to around 3470, averaging around 3400 with a couple of runs. I was, with the same kernel and Chenglu's data2loop patch, getting in the 3200s to 3300s, so the difference seemed somewhat significant to me.
Unfortunately, it seems to puke out every time I reboot. The data must get screwed up somehow in the /data/ext4 files, and my G Tab goes schizo and won't even finish the bootup process - screen flickers from the "Vegan" logo during boot, and then screen shuts off. Have to hard shut-down then hard reboot to recovery and wipe data to get things working again.
The /misc/inandop.log file seems to report that data-to-loop was successfully enabled, so I'm not sure exactly why it's puking out in such epic fashion. Have to work more on this later.
Click to expand...
Click to collapse
So just to update this thread, I did get ext4 working stably on the /data partition. The trick is doing everything from recovery mode - making the ext4 filesystem via adb push and adb shell, then flashing an update.zip from recovery that packages a kernel that supports ext4 with a ramdisk/init.rc that mounts /data as ext4 and skips the inandop.sh step entirely since that was the source of bootloops (stock inandop.sh expects /data to be ext3 and pukes if it's not).
You can see my detailed instructions over here: http://forum.xda-developers.com/showpost.php?p=10489732&postcount=23
The bad news is really no significant performance bump from ext4 in the standard 2gig partition. But I expect it to be stable as hell. Still need to experiment more with ways to speed things up (safely).
Hey all, This is my first post on the forum, so I thought I'd make it a guide.
I've run into a few roadblocks with installing some development tools like python and stuff, so I figured I'd try and make a filesystem image to hold all of it, what with Android preventing executables on your SD card from actually executing. You should NOT try and use this for freeing up space by moving apps to it. That just breaks things.
Things to know
A mount point is where you access the contents of a filesystem through.
A filesystem is basically a virtual disk.
I'm not responsible for your phone melting into a pool of acid or anything else you want to blame me for. You executed the commands, not me.
If there's been guides for this already, I didn't find them.
Playing with your system files is dangerous. Be careful
Playing with filesystems is even MORE dangerous. Don't do it unless you know what's going on.
Reversing and then combining two words in a sentence, then calling it a bit more dangerous is incredibly dangerous and can kill you. Don't do it.
As I said earlier: This is not intended to free up internal storage on your phone. It is intended to create extra space to put command-line executables.
This was created for those of us who have older phones that don't have a whole lot of internal storage. (Like the Milestone) This should also work on anything that meets the requirements listed below. I use it on a Motorola Milestone A853 running CyanogenMod 7.2.4f UMTS/SHOLES, and it took some tweaking, but it worked. Commands you put into your shell are marked with `bolded text and surrounded by back-ticks. Leave the back-ticks out. (`)`
Requirements! (obtained using `which <command name>` or a specified command)
Free space on your mSD
losetup (configuring loop devices)
mknod (creating loop devices)
mount (mounting filesystems)
mkdir (making directories)
su (root access)
umount (unmounting)
mke2fs (formatting)
Linux filesystem support (`cat /proc/filesystems | grep ext`)
Here we go!
This assumes you are going to use an ext4-formatted filesystem image mounted on /data/development/, with the image located at /mnt/sdcard/development.img. You WILL need to determine your phone's supported ext<num> filesystems using `cat /proc/filesystems | grep ext`. Choose the highest-numbered one.
Create the mountpoint, the image, and then format the image to ext4.
`mkdir /data/development/``dd if=/dev/zero of=/mnt/sdcard/development.img count=409600 bs=1024``mke2fs -l /mnt/sdcard/development.img -b 1024 -t ext4`
Create the block device, set it to point at the image, then mount it.
`mknod /dev/block/loop250 b 7 250``losetup /dev/block/loop250 /mnt/sdcard/development.img``mount -t ext4 /dev/block/loop250 /data/development`
Test it to see if it works.
`touch /data/development/testfile;ls /data/development/`
Use it.
Unmounting (/!\ For connecting to a computer to avoid corruption /!\)
`umount /mnt/sdcard/development.img``losetup -d /dev/block/loop250`
Well, that's all I've got. Suggestions and corrections, as well as good feedback are appreciated.
Some PAQs (Possibly Asked Questions)
Q: I'm getting some errors. Here they are: pastebin.com/some-errors
A: Great, I'll see what I did wrong, and if it's something on your end, I'll try and help you out.
Q: It dun werk
A: Tell me the errors instead of complaining about them. Just saying that does not help me and wastes my time and bandwidth.
Q: "No space left on device"; "Operation not permitted"; "No such file or directory"
A: SD card full; not root (or SD card is full, or something broke); Mount point doesn't exist or (if 'mount' returns that and the point exists) something not cool happened.
Usage guide on Non-Android devices
Some interesting bits of information for use on computers:
There's probably not an easy way to read/write/mount EXT2 filesystems on a Windows machine, so if you have a solution that's stable, go for it.
Linux computers definitely should support this, and I'm not sure about OSX computers, but in theory would work if you had the right things installed. iPhones ... well I have no idea what you'd do with it there.
Linux:
On the device:
THIS PART IS EXTREMELY IMPORTANT, NOT DOING THIS WILL RESULT IN BROKEN FILESYSTEMS!
`umount /mnt/sdcard/development.img`
`losetup -d /dev/block/loop250`Plug your phone into your computer, making sure that the computer mounts it and reads it as removable media, and that the image is present.
On the computer: (Path of your user folder, and the mount point at which your device's SD is mounted can and very likely will vary from distribution to distribution. Check your paths.)
`mkdir /home/user/android-devtools/`
`mount -o loop /media/user/android-sd/development.img /home/user/android-devtools/`Now, you can change things within the image as you normally would a file or folder. Be warned, sometimes doing this can create conflicts in the file permissions, so running the right commands on the phone to change the permissions as root (use `chmod` for the permissions) can save you boatloads of trouble getting things to work, or editing files.
Mac OSX might or might not be the same, but seeing as I haven't picked up an Apple computer since maybe 5 years ago, let alone attempted filesystem management, I just don't know.
Last night I started to run e2fsck on my /data partition to do a bit of investigation to see if had any corruption (again)
Anyway I unmounted /data in TWRP and went to the terminal and issued the following command:
e2fsck -fvyD /dev/block/mmcblk0p8
12 hours later and it was still running mainly reporting "Clone multiply-claimed blocks? Yes" Yes is because I have y flag turned on to answer yes to all questions.
So I killed it as it has never run this long before (previously 3 hours max).
I then formatted data which I thought would make it unnecessary and ran the command again after unmounting data and it is doing the same thing.....
Why?
Code:
.
.
.
(
inode
#
869248
. mod time
Sat Aug 25 18:23:27 2012
)
Clone multiply-claimed blocks? Yes
It appears to be doing this for many (if not all) inodes
Over and over.
This makes it seem like my /data partition is very corrupted and also Aug 25 is when I bought the device in 2012 so it has gone right back to the start.
I suspect it is somehow related to turning data journaling off whilst testing hunds kernels.
What is the definitive command to ensure data journaling is on please?
sbdags said:
I then formatted data which I thought would make it unnecessary and ran the command again after unmounting data and it is doing the same thing.....
Click to expand...
Click to collapse
That's strange - after formatting a partition, e2fsck must run in seconds and without errors, and it should definitely not bring up anything related to the old data. How did you format it?
_that said:
That's strange - after formatting a partition, e2fsck must run in seconds and without errors, and it should definitely not bring up anything related to the old data. How did you format it?
Click to expand...
Click to collapse
I formatted it using the big format data button in TWRP but yes I agree it doesn't appear to have done what would call a format? Maybe the equivalent of a quick format? How do I properly format the mmcblk0p8 partition?
If I just issue e2fsck -n /dev/block/mmcblk0p8 it comes back with clean. It is only when I force a scan that it finds all these errors.
Just reformatted and run again and it is dong the same again. ??
Btw what is the command to turn data journaling on? Or once formatted does journaling reset itself?
sbdags said:
Just reformatted and run again and it is dong the same again. ??
Btw what is the command to turn data journaling on? Or once formatted does journaling reset itself?
Click to expand...
Click to collapse
Can you find the following line in your recovery log? This would be the command that TWRP used to format.
Code:
make_ext4fs command:
Normally, formatting ext4 with default options enables the journal.
_that said:
Can you find the following line in your recovery log? This would be the command that TWRP used to format.
Code:
make_ext4fs command:
Normally, formatting ext4 with default options enables the journal.
Click to expand...
Click to collapse
It reports the following:
Code:
make_ext4fs -l -16384 -a /data /dev/block/mmcblk0p8
sbdags said:
It reports the following:
Code:
make_ext4fs -l -16384 -a /data /dev/block/mmcblk0p8
Click to expand...
Click to collapse
Looks plausible. And look what I've found:
http://sourceforge.net/p/e2fsprogs/bugs/292/
You could try running the make_ext4fs command manually with the "-w" (wipe) option added - if I understand the code correctly, that should issue discard requests for all blocks, similar to fstrim.
sbdags said:
I formatted it using the big format data button in TWRP but yes I agree it doesn't appear to have done what would call a format? Maybe the equivalent of a quick format? How do I properly format the mmcblk0p8 partition?
If I just issue e2fsck -n /dev/block/mmcblk0p8 it comes back with clean. It is only when I force a scan that it finds all these errors.
Just reformatted and run again and it is dong the same again. ??
Btw what is the command to turn data journaling on? Or once formatted does journaling reset itself?
Click to expand...
Click to collapse
I mentioned this during the fiasco we had in the cromi thread awhile back, but I think it got lost in the rush of comments, let me know if you get it fixed, would love to be able to complete a check.
faustus1005 said:
I mentioned this during the fiasco we had in the cromi thread awhile back, but I think it got lost in the rush of comments, let me know if you get it fixed, would love to be able to complete a check.
Click to expand...
Click to collapse
Yeah something is not quite right with running fsck on the TF700. Oddly on my prime it runs without issue. I wonder if Asus has changed the way inodes are mapped. I know Samsung do this. I wonder if contributes to any lag as well?
_that said:
Looks plausible. And look what I've found:
http://sourceforge.net/p/e2fsprogs/bugs/292/
You could try running the make_ext4fs command manually with the "-w" (wipe) option added - if I understand the code correctly, that should issue discard requests for all blocks, similar to fstrim.
Click to expand...
Click to collapse
Let me do another backup and try the -w command as well. It's quite frustrating as I wonder if being able to run fsck correctly would help solve some of the lag issues we see.
_that said:
Looks plausible. And look what I've found:
http://sourceforge.net/p/e2fsprogs/bugs/292/
You could try running the make_ext4fs command manually with the "-w" (wipe) option added - if I understand the code correctly, that should issue discard requests for all blocks, similar to fstrim.
Click to expand...
Click to collapse
Adding the -w command fixed it (I think) e2fsk now completes in about a minute with no errors
Problem is now when I check if data journaling is on with a mount command there is no indication that it is - how to turn it off?
This is the command issued that turned it off I believe it's the "-O" that turns the journaling off:
Code:
run_program("/tmp/tune2fs.ext4", "-O", "^has_journal", "-c", "1", "-i", "1d", "-m", "0", "-o", "^journal_data_writeback", "/dev/block/mmcblk0p8");
is it as easy as issuing the same command without the "-O", "^has_journal" to get it back on?
OK I found some info looks like I need to issue this command in the installer?
Code:
run_program("/tmp/tune2fs.ext4", "-O", "+has_journal", "-c", "5", "-i", "5d", "-m", "0", "-o", "^journal_data_ordered", "/dev/block/mmcblk0p8");
I'll try that out now
Thanks for all the help @_that - I'm slowly getting there I think. :good:
sbdags said:
Last night I started to run e2fsck on my /data partition to do a bit of investigation to see if had any corruption (again)
Anyway I unmounted /data in TWRP and went to the terminal and issued the following command:
e2fsck -fvyD /dev/block/mmcblk0p8
12 hours later and it was still running mainly reporting "Clone multiply-claimed blocks? Yes" Yes is because I have y flag turned on to answer yes to all questions.
So I killed it as it has never run this long before (previously 3 hours max).
I then formatted data which I thought would make it unnecessary and ran the command again after unmounting data and it is doing the same thing.....
Why?
Code:
.
.
.
(
inode
#
869248
. mod time
Sat Aug 25 18:23:27 2012
)
Clone multiply-claimed blocks? Yes
It appears to be doing this for many (if not all) inodes
Over and over.
This makes it seem like my /data partition is very corrupted and also Aug 25 is when I bought the device in 2012 so it has gone right back to the start.
I suspect it is somehow related to turning data journaling off whilst testing hunds kernels.
What is the definitive command to ensure data journaling is on please?
Click to expand...
Click to collapse
You can try this.
tune2fs -l /dev/block/mmcblk0p8
You should see your system mounting has a "has_journal".
If it is not there, then your journal is off.
To turn it on.
tune2fs -O +has_journal /dev/block/mmcblk0p8
You should use the data_write for your journal to improve performance.
If you do a clean installation for your rom, the e2fsck should run less than a minute if you have a minimal apps installed. However, if you restore a nandroid backup for your system, it will take forever even though you don't have any app. I don't know why it does that with the nandroid backup..
I think that when you use e2fsck with a nandroid restore from your backup, the e2fsck is trying to convert your ext4 system to ext3 system.. It is just my guess. Maybe you and _that can figure why it does that.. Good luck...:fingers-crossed:
LetMeKnow said:
You can try this.
tune2fs -l /dev/block/mmcblk0p8
You should see your system mounting has a "has_journal".
If it is not there, then your journal is off.
To turn it on.
tune2fs -O +has_journal /dev/block/mmcblk0p8
You should use the data_write for your journal to improve performance.
If you do a clean installation for your rom, the e2fsck should run less than a minute if you have a minimal apps installed. However, if you restore a nandroid backup for your system, it will take forever even though you don't have any app. I don't know why it does that with the nandroid backup..
I think that when you use e2fsck with a nandroid restore from your backup, the e2fsck is trying to convert your ext4 system to ext3 system.. It is just my guess. Maybe you and _that can figure why it does that.. Good luck...:fingers-crossed:
Click to expand...
Click to collapse
Interesting, however, restored a nandroid and reran e2fsck and it finished in 30 secs with no errors Installed my ROM and ran it again and same again so it seems my problems are resolved for now
Thanks for the tips on data journaling.
BTW can you reup or send me the latest ET please - ready and confident to test it again now
sbdags said:
Interesting, however, restored a nandroid and reran e2fsck and it finished in 30 secs with no errors Installed my ROM and ran it again and same again so it seems my problems are resolved for now
Thanks for the tips on data journaling.
BTW can you reup or send me the latest ET please - ready and confident to test it again now
Click to expand...
Click to collapse
Haha, I found out that when I restore a good backup from nandroid and have more than 50% of problem with e2fsck.
I am still on a business trip. Let see if my thumb drive has the latest ET. Otherwise, I have it on my personal laptop at home. I will let you know in bit..
Woke up this morning turned on my slide with the wonderful KKDS 1.3 from Emmanuel U, dont think this is a rom issue at all as have been using this rom for a very long while.
Anyhow so i get this "
System UIDs Inconsistent" Error popup that reads :
-------------------------
System UIDs Inconsistent
UIDs on the System are Inconsistent you need to wipe the Data partition or your device will be unstable
<im feeling lucky button>
----------------------------------
so whats the exact cause of the issue, if i re flash my twrp backup will this fix my problem? why does this issue arise & is there any other way to fix?
steve_77 said:
Woke up this morning turned on my slide with the wonderful KKDS 1.3 from Emmanuel U, dont think this is a rom issue at all as have been using this rom for a very long while.
Anyhow so i get this "
System UIDs Inconsistent" Error popup that reads :
-------------------------
System UIDs Inconsistent
UIDs on the System are Inconsistent you need to wipe the Data partition or your device will be unstable
<im feeling lucky button>
----------------------------------
so whats the exact cause of the issue, if i re flash my twrp backup will this fix my problem? why does this issue arise & is there any other way to fix?
Click to expand...
Click to collapse
I wonder if it is talking about different owners of files in /data...
...unless UID means something different than User Identification. I just opened a terminal emulator (on TBalden's 10.1) and did:
ls -l /data
ls -l /data/data
This shows all the folders and files in the respective directories with the assigned UID and GID (Group Identification). There was no consistent pattern. Even in /data/data, it looks like there was a different user assigned to every folder.
Which unfortunately means the easy way, 'chown -R root:root /data' is out.
Probably the cleanest solution HERE
Lanchon REPIT: The In-Device Data-Sparing Repartitioning Tool For Android
REPIT is a simple, safe, device-only, data-sparing, and easily portable repartitioning tool for Android devices:
device-only: just flash a zip file in recovery to repartition the device.
simple: rename the zip file before flashing to configure your choice of partition sizes, file systems, wipes, etc.
safe:
a correctly ported REPIT can never hard-brick your device.
before starting, REPIT checks for the existence of all the tools that will be needed for the task at hand, verifies that the current partition layout passes several sanity checks, checks and fixes all the involved file systems, and verifies that the new partition layout will meet its sanity checks too. REPIT performs a dry-run of the complete repartitioning process to detect possible problems early on.
if REPIT fails, it will nonetheless try to restore your device to a working in-between state.
my estimate is that around 1000 users have already repartitioned with REPIT with no incidents of data loss, boot loops or soft bricks (2016-04).
easily portable: a simple configuration file is all that is needed to port REPIT to a new device.
Documentation (HELL YEAH, read this!) -> HERE
Downloads -> HERE
this is a user discussion thread. post here for general discussion and user support.
for official support create a Github issue, but only of you have read the docs.
new versions will not be announced here.
please 'watch' the Github project to receive notifications.
special thanks to the.gangster.
XDA:DevDB Information
Lanchon REPIT, Tool/Utility for all devices (see above for details)
Contributors
Lanchon
Source Code: https://github.com/Lanchon/REPIT
Version Information
Status: Stable
Created 2016-04-13
Last Updated 2019-06-07
Reserved
Reserved
Good to have a place again to discuss issues or simple questions.
Great tool, that meanwhile left its corner where it grew up (the Galaxy S2 - i9100), meanwhile supporting a growing number of devices.
RELEASES AND CHANGELOG
the.gangster said:
Good to have a place again to discuss issues or simple questions.
Great tool, that meanwhile left its corner where it grew up (the Galaxy S2 - i9100), meanwhile supporting a growing number of devices.
@Lanchon
you accidently linked the downloads to your flashize github instead of the Repit one.
Click to expand...
Click to collapse
lol thanks, mucho copy/pasta, fixed!
Thanks so much!
Perfect tool, used this evening on a old but still used I9100!
Tried this on a n7000 recently, tried to resize cache, which didn't work, saw partition structure with modem after cache, and understand your concerns in the comments. I imagine the we could so some experiments like you are doing for the Honor 4, however I'm not at the stage where I want to volunteer my device as potential sacrifice.
A question on "wipe", how sure are we that "wipe" actually wipes, ie is all the previous data non-recoverable down to a block level?
Does the wipe trigger some type of NAND erase cycle? This would be problematic on n7000 given the hard brick issues with NAND firmware there, and on the n7000 IIRC the ioctl() is filtered and purposely ignored, maybe this is also an issue with i9100? I don't recall.
If I have a used ext4 parition, and then format & wipe it then I think a fstrim happens. If I then change it to vfat, and dd the partition, am I able to recover blocks from the original ext4 partition, or does the trim cause the nand to regard the blocks as empty and return 0x0000000 or 0x11111111 or something deterministic?
I'm looking for a wipe that actually wipes, ext4 and vfat, rather then my current solution which is to dd if=/dev/zero of=/path, which is slow and is difficult to get a progress report, as kill -USR1 is a PITA.
samarium said:
Tried this on a n7000 recently, tried to resize cache, which didn't work, saw partition structure with modem after cache, and understand your concerns in the comments. I imagine the we could so some experiments like you are doing for the Honor 4, however I'm not at the stage where I want to volunteer my device as potential sacrifice.
A question on "wipe", how sure are we that "wipe" actually wipes, ie is all the previous data non-recoverable down to a block level?
Does the wipe trigger some type of NAND erase cycle? This would be problematic on n7000 given the hard brick issues with NAND firmware there, and on the n7000 IIRC the ioctl() is filtered and purposely ignored, maybe this is also an issue with i9100? I don't recall.
If I have a used ext4 parition, and then format & wipe it then I think a fstrim happens. If I then change it to vfat, and dd the partition, am I able to recover blocks from the original ext4 partition, or does the trim cause the nand to regard the blocks as empty and return 0x0000000 or 0x11111111 or something deterministic?
I'm looking for a wipe that actually wipes, ext4 and vfat, rather then my current solution which is to dd if=/dev/zero of=/path, which is slow and is difficult to get a progress report, as kill -USR1 is a PITA.
Click to expand...
Click to collapse
hi,
well, it's not that it didn't work lol! cache was intentionally left out of the resizable set. you've seen the comments and the 4X discussion so you know exactly why.
i wouldn't do the experiments anyway. in the Honor there is something useful to be gained: 1 extra GB to a 4GB /data, which is a lot. in the S2 cache is 100MB. cache is unused nowadays, so the only logical thing to do is shrink it. plus you want something left, like 32MB. so we will risk devices for 68MB? not on my watch
secure wipe is a very complex issue and there's an awful big can of worms there, believe me. it'll try to paint a picture.
ssds have a bunch of 'trim' commands. there's ERASE and DISCARD, and later TRIM was added, all with slightly different semantics. ans they all come in standard and secure flavors. most recently you can come across the standard non-queued commands or the new queued variants.
the non-secure flavored commands are performance-oriented: they are hints to help the ssd. the drive can ignore them if it wants. it can ignore requests for short ranges but honor requests for large extents, or whatever.
more recently devices can implement DRAT and RZAT (deterministic read after trim and read zero after trim), ending the previous non-deterministic behavior. take a look:
http://www.t13.org/documents/uploadeddocuments/docs2010/e09158r2-trim_clarifications.pdf
but there are a million bugs in ssd firmware so linux blacklists broken functionality. like RZAT in many drives (you can google it), even if the drive says it supports it.
in any case RZAT drives just alter the FTL (flash translation layer) structures to unmap the block, they don't erase flash, so your data is typically still there but normally inaccessible. and there is probably even a history of previous snapshots of the block, not just the latest.
that's where the secure commands enter the picture: supposedly the firmware must actually erase from flash all copies of the targeted blocks. in reality, firmwares are so buggy i wouldn't trust them for that, except maybe if the drive is intel brand.
back to the real world... the exynos:
>Does the wipe trigger some type of NAND erase cycle? This would be problematic on n7000 given the hard brick issues with NAND firmware there, and on the n7000 IIRC the ioctl() is filtered and purposely ignored, maybe this is also an issue with i9100? I don't recall.
you are living in the past. more than two years ago i started arguing that non-secure trim commands should in all probability work fine and could be used to speed up the S2 family. rationale is complex but you can google my threads about it. no-one listened so 2 years ago i learned to build kernels and did some tests with volunteers and began publishing trimming kernels (google i9100 trim for a bit of history). finally in 2016 trim was finally added to official CM 13, 12.1 and 11.
you could be trimming your device without knowing: if you are using recent official CM, you are already trimming. many other kernels switched to trim too.
but... in the S2, secure trim kills the eMMC, so only non-secure variants are issued by trim kernels. secure erase is not a possibility.
i don't know if the eMMC is RZAT, but i wouldn't trust the firmware anyway. if you want to wipe, dd is your best option. but don't use zeros!!! some ssd's compress data. use a pseudo random stream instead. after all, non-secure trim is not very different from a write, in the best case, so just write for safety and be done with it.
yes, dd won't wipe reserved blocks. no way to do that except:
-MAYBE... an eMMC full reset (including bootloader) by resizing the boot partitions (google) which is very dangerous and can easily hard brick the device.
-putting a power drill to you beloved S2
regarding what repit does:
-it trims only if the kernel has trim enabled.
-and it trims only ext4 partitions.
thanks!
Thanks for the detailed reply.
I'm currently using i9100 CM12.1 & n7000 CM13/Nightowl, and have been trimming vfat partitions.
Now you remind me, I do recall reading threads about trim safe a while ago, but I think I was more irritated at the time by the seemingly random kernel crashes & process terminations that eventually were tracked down to kernel interrupt fp register save/restore, and went down the Sony path for a while while the n7000 was unusable. Amusingly, the n7000 now has CM13, while the Sony Z Ultra is languishing and isn't looking good for CM13 at the moment.
While I agree that for general SSDs, zero blocks could be compressed, and blocks could be deduped, do you really think this is happening in n7000/i9100 emmc? I dont expect so, seems they seem too old. I agree that a /dev/urandom would be a better if= than /dev/zero for dd, however this is going to use much more power, and I'm usually wiping a phone with limited time and power.
I'm not trying to protect from something as serious as direct flash chip access, just want to be reasonably sure that whatever was on the phone has been overwritten and is not easily available to standard and even mildly esoteric extraction methods.
samarium said:
Thanks for the detailed reply.
I'm currently using i9100 CM12.1 & n7000 CM13/Nightowl, and have been trimming vfat partitions.
Now you remind me, I do recall reading threads about trim safe a while ago, but I think I was more irritated at the time by the seemingly random kernel crashes & process terminations that eventually were tracked down to kernel interrupt fp register save/restore, and went down the Sony path for a while while the n7000 was unusable. Amusingly, the n7000 now has CM13, while the Sony Z Ultra is languishing and isn't looking good for CM13 at the moment.
While I agree that for general SSDs, zero blocks could be compressed, and blocks could be deduped, do you really think this is happening in n7000/i9100 emmc? I dont expect so, seems they seem too old. I agree that a /dev/urandom would be a better if= than /dev/zero for dd, however this is going to use much more power, and I'm usually wiping a phone with limited time and power.
I'm not trying to protect from something as serious as direct flash chip access, just want to be reasonably sure that whatever was on the phone has been overwritten and is not easily available to standard and even mildly esoteric extraction methods.
Click to expand...
Click to collapse
ok. then i would format in ext4 and trim. you can use repit for doing both in a single shot, but you need a trimming kernel. take a dd snapshot of one or a couple of short pieces of the partition, both before and after the operation. compare them and post here, i'd surely like to know .
but... keep in mind that there are several different emmcs out there in the S2s, so you'd need to validate for each one. which reminds me, also post your device emmc IDs! use the eMMC brickbug check app for instance.
regarding the /dev/urandom vs /dev/zero thing, i'm not religious, i don't hold unwarranted beliefs about what i don't know. i investigate, i have no gut feelings on stuff
but if you want to know: i'm a bit obsessive, i certainly would have done at least something if i was coding the FTL, even if it is old. if your design is RZAT, the least you can do is trim the block instead of writing it when data is all zeros. you may think this would slow the emmc down, but this can be implemented in pseudo-zero time: writing usually entails generating some kind of checksum of the data. so note the magic checksum that corresponds to an all-zero block, and only if the checksum of the data being written matches the magic, actually check for all-zero data before writing. of course this can be 'attacked' by intentionally generating checksum collisions but who cares. the time saved by not garbage collecting, not using a free block, and not writing the block with all-zeros certainly pays for the checksum comparison against the magic value. also note that straight brute force comparison of all blocks is mostly ok too, as most written blocks will fail early anyway (ie, after one or a couple of bytes).
of course The One True Solution(R) is to encrypt the phone before using it...... assuming that you can actually wipe the key when you need to lol. ideally the key is in a separate hardware keystore adept at wiping keys. but on the S2 it just in the same eMMC . so this is a bit like explaining where the universe came from by saying that god created it, and then conveniently forget to explain where did god come from lol.
the power drill solution still mostly works on the S2 though
Pls help
##################################
Lanchon REPIT
A Data-Sparing Repartitioning Tool
Version: 2016-04-15
Device: i9100
Copyright 2016, Lanchon (GPLv3)
####################################
===== PRELIMINARY CHECKS =====
info: valid package names: <prefix>[-(system|data|sdcard|preload)=<conf>]...<suffix>
info: valid partition <conf> values: [<size-in-GiB>|same|min|max][+[keep|wipe][+[ext4|vfat|f2fs|swap|raw]]]
----- DEFAULTS -----
system = size:same + content:keep + fs:ext4
data = size:same + content:keep + fs:ext4
sdcard = size:same + content:keep + fs:vfat
preload = size:same + content:keep + fs:ext4
info: parsing package name
----- CONFIGURATION -----
system = size:1.0 + content:keep + fs:ext4
data = size:6 + content:keep + fs:ext4
sdcard = size:max + content:keep + fs:vfat
preload = size:min + content:keep + fs:ext4
info: disabling swap
BusyBox v1.22.1 bionic (2016-01-10 12:49 +0530) multi-call binary.
Usage: dirname FILENAME
Strip non-directory suffix from FILENAME
info: copying package to '/tmp'
cp: '/tmp/lanchon-repit-20160415-system=1.0-data=6-sdcard=max-preload=min wipe-i9100.zip' and '/tmp/lanchon-repit-20160415-system=1.0-data=6-sdcard=max-preload=min wipe-i9100.zip' are the same file
[ERROR 1]
Whats wrong
just change your current directory to /tmp and try to flash it again from there. (all within the recovery)
Lanchon said:
ok. then i would format in ext4 and trim. you can use repit for doing both in a single shot, but you need a trimming kernel. take a dd snapshot of one or a couple of short pieces of the partition, both before and after the operation. compare them and post here, i'd surely like to know .
but... keep in mind that there are several different emmcs out there in the S2s, so you'd need to validate for each one. which reminds me, also post your device emmc IDs! use the eMMC brickbug check app for instance.
regarding the /dev/urandom vs /dev/zero thing, i'm not religious, i don't hold unwarranted beliefs about what i don't know. i investigate, i have no gut feelings on stuff
Click to expand...
Click to collapse
bauner produced a new n7000 IsoRec with TWRP + OTG which I wanted, so while I was installing I tested a few things.
I generated a block binary file with each of the 1kB blocks annotated for the 200MB n7000 cache partition, and dd'd the file to the cache partition prior to each mkfs.
Code:
$ cat blocknull.pl
#!/usr/bin/perl
use strict;
my $count = $ARGV[0];
$count = 1 if $count == 0;
$count *= 1024 if $ARGV[0] =~ /M$/;
$count *= 1024 * 1024 if $ARGV[0] =~ /G$/;
for (my $i = 1; $i <= $count; $i++)
{
printf "\nBLOCK %10d kB %6d MB %2d GB BLOCK\n", $i, $i / 1024, $i / 1024 / 1024;
print chr(0) x (1024-43);
}
$
Code:
BLOCK 1 kB 0 MB 0 GB BLOCK
...
BLOCK 204800 kB 200 MB 0 GB BLOCK
Then operated on both vfat and ext4 file systems to determine which of the 204800 block signatures where remaining after various operations.
Code:
# tr -d '\0' < /dev/block/mmcblk0p7 | grep BLOCK.*BLOCK > cache.x.y.desc
...
# wc -l cache.?.?.*
204800 cache.1.1.dd
201632 cache.1.2.mkfs.fat32
197041 cache.1.3.copyfile
204800 cache.2.1.dd
0 cache.2.2.mkfs.ext4
204800 cache.3.1.dd
197380 cache.3.2.mkfs.ext4.twrp
60 cache.3.3.fstrim
60 cache.3.4.copyfile
#
So as expected vfat format doesn't change the underlying data returned, it just gets changed as it is used, as show by the signature counts on cache.1.*
ext4 is more interesting, as I tried a manual mke2fs -t ext4 for cache.2.* and TWRP wipe ext4 for cache.3.*. The TWRP wipe is not using secure discard as can be seen from recovery.log, but things are largely back on track after the first fstrim except for a hole from 131081 kB to 131125 kB.
Code:
recovery.log:
warning: wipe_block_device: Wipe via secure discard suppressed due to bug in EMMC firmware
Maybe more interesting is that even though it seems manual mke2fs -t ext4 does a wipe, when I do a fstrim is still says 180MB trimmed, which I don't expect, some inconsistency in fs metadata setup in mkfs maybe? Doesn't matter to me, 1 or 2 wipes, but seems I will be using mke2fs -t ext4 via terminal and then using TWRP wipe/format as required.
FYI, EMMC IDs PNG attached.
samarium said:
bauner produced a new n7000 IsoRec with TWRP + OTG which I wanted, so while I was installing I tested a few things.
I generated a block binary file with each of the 1kB blocks annotated for the 200MB n7000 cache partition, and dd'd the file to the cache partition prior to each mkfs.
Code:
$ cat blocknull.pl
#!/usr/bin/perl
use strict;
my $count = $ARGV[0];
$count = 1 if $count == 0;
$count *= 1024 if $ARGV[0] =~ /M$/;
$count *= 1024 * 1024 if $ARGV[0] =~ /G$/;
for (my $i = 1; $i <= $count; $i++)
{
printf "\nBLOCK %10d kB %6d MB %2d GB BLOCK\n", $i, $i / 1024, $i / 1024 / 1024;
print chr(0) x (1024-43);
}
$
Code:
BLOCK 1 kB 0 MB 0 GB BLOCK
...
BLOCK 204800 kB 200 MB 0 GB BLOCK
Then operated on both vfat and ext4 file systems to determine which of the 204800 block signatures where remaining after various operations.
Code:
# tr -d '\0' < /dev/block/mmcblk0p7 | grep BLOCK.*BLOCK > cache.x.y.desc
...
# wc -l cache.?.?.*
204800 cache.1.1.dd
201632 cache.1.2.mkfs.fat32
197041 cache.1.3.copyfile
204800 cache.2.1.dd
0 cache.2.2.mkfs.ext4
204800 cache.3.1.dd
197380 cache.3.2.mkfs.ext4.twrp
60 cache.3.3.fstrim
60 cache.3.4.copyfile
#
So as expected vfat format doesn't change the underlying data returned, it just gets changed as it is used, as show by the signature counts on cache.1.*
ext4 is more interesting, as I tried a manual mke2fs -t ext4 for cache.2.* and TWRP wipe ext4 for cache.3.*. The TWRP wipe is not using secure discard as can be seen from recovery.log, but things are largely back on track after the first fstrim except for a hole from 131081 kB to 131125 kB.
Code:
recovery.log:
warning: wipe_block_device: Wipe via secure discard suppressed due to bug in EMMC firmware
Maybe more interesting is that even though it seems manual mke2fs -t ext4 does a wipe, when I do a fstrim is still says 180MB trimmed, which I don't expect, some inconsistency in fs metadata setup in mkfs maybe? Doesn't matter to me, 1 or 2 wipes, but seems I will be using mke2fs -t ext4 via terminal and then using TWRP wipe/format as required.
FYI, EMMC IDs PNG attached.
Click to expand...
Click to collapse
>it seems manual mke2fs -t ext4 does a wipe
it shouldn't. mke2fs -i thought- doesn't trim unless invoked with a specific option. repit invokes with the flag:
https://github.com/Lanchon/REPIT/blob/master/repit-fs-ext4.sh#L51
as to the question of why your markers were cleared anyway, i've no answer.
>even though it seems manual mke2fs -t ext4 does a wipe, when I do a fstrim is still says 180MB trimmed
some kernels inform the newly trimmed space: if you run trim twice you'll get "0 bytes trimmed" the second time.
other kernels (like CM's i9100) always inform (and presumably trim) the full partition free space, so 180MB is ok. if you run trim twice you'll get the same number of bytes both times.
>things are largely back on track after the first fstrim except for a hole from 131081 kB to 131125 kB.
here are some candidate causes:
a) ext4 fs is based on 'blocks' with each block being a constant number of sectors. it makes sense that fstrim should work on whole blocks. also, when ext4 writes a 'short block', ie one not completely filled with data, some sectors and the end of the block might remain unused. it makes sense for ext4 not to write these sectors at all. but the complete block will be marked as used given that the block is the granularity for that, so no sector in the block will be trimmed by fstrim. so some pre-format data might survive in these short blocks after fstrim by this mechanism, which could explain what you observed.
b) another cause could be that the emmc might ignore trim requests for spans shorter than some size or for misaligned spans (remember that erases are performed in large chunks) giving rise to the observed data survival. when formatting no data survives because partitions are position- and size-aligned to 1MB or 4MB, which might be enough to guarantee trim. (i find this explanation unlikely, but who knows.)
about repit:
repit formats and trims ext4 and f2fs in a single operation.
to wipe /data a single repit should be enough:
-data=+wipe
to wipe /sdcard you can take advantage of repit's capability to convert file systems and do two repits:
-sdcard=+wipe+ext4
-sdcard=+wipe
an idea that comes out of this is that repit could temporarily format in ext4 before formatting in other modes that do not support format-time trimming, such as when wiping vfat and swap. this would make the above 2 steps unnecessary.
darkz16 said:
[...]
info: disabling swap
BusyBox v1.22.1 bionic (2016-01-10 12:49 +0530) multi-call binary.
Usage: dirname FILENAME
Strip non-directory suffix from FILENAME
info: copying package to '/tmp'
cp: '/tmp/lanchon-repit-20160415-system=1.0-data=6-sdcard=max-preload=min wipe-i9100.zip' and '/tmp/lanchon-repit-20160415-system=1.0-data=6-sdcard=max-preload=min wipe-i9100.zip' are the same file
[ERROR 1]
Click to expand...
Click to collapse
this is very interesting.
the relevant lines are these:
https://github.com/Lanchon/REPIT/blob/master/repit.sh#L318-L322
Code:
if [ -f "$packageName" ]; then
if [ "$(dirname $(readlink -f "$packageName"))" != "/tmp" ]; then
info "copying package to '/tmp'"
cp -f "$packageName" "/tmp/"
hint="this package copied itself to '/tmp'; please run it again from there"
the line:
if [ -f "$packageName" ]; then
assures that the file $packageName exists.
but these logfile lines:
Code:
BusyBox v1.22.1 bionic (2016-01-10 12:49 +0530) multi-call binary.
Usage: dirname FILENAME
Strip non-directory suffix from FILENAME
are proof that 'dirname' is being invoked without argument, which means that 'readlink -f "$packageName"' produced an error!
how can 'readlink -f <file>' (which means 'canonicalize') give an error when invoked on an existing file is beyond me. it looks like your implementation of 'readlink' is broken.
what exact recovery are you using? who made it or where is it being distributed?
EDIT:
btw there is a quoting error in the code here but it shouldn't affect this. the line should be:
if [ "$(dirname "$(readlink -f "$packageName")")" != "/tmp" ]; then
darkz16 said:
[...]
cp: '/tmp/lanchon-repit-20160415-system=1.0-data=6-sdcard=max-preload=min wipe-i9100.zip' and '/tmp/lanchon-repit-20160415-system=1.0-data=6-sdcard=max-preload=min wipe-i9100.zip' are the same file
[ERROR 1]
Whats wrong
Click to expand...
Click to collapse
so i committed what i think is a fix for this problem that i cant reproduce:
https://github.com/Lanchon/REPIT/commit/c7bc3655acc89ee554d1dafe09c7bb9948483c03
and attached is build for you to try.
IMPORTANT: EXTRACT THE ZIP FIRST, DONT JUST FLASH IT!
Lanchon said:
so i committed what i think is a fix for this problem that i cant reproduce:
https://github.com/Lanchon/REPIT/commit/c7bc3655acc89ee554d1dafe09c7bb9948483c03
and attached is build for you to try.
IMPORTANT: EXTRACT THE ZIP FIRST, DONT JUST FLASH IT!
Click to expand...
Click to collapse
Hi Lanchon.
I'm installed this zip but i have the same problem.
I'm copy, before zipping the achive, on the \tmp directory of emmc
What have i do wrong?.
Thanks.
chapito said:
Hi Lanchon.
I'm installed this zip but i have the same problem.
I'm copy, before zipping the achive, on the \tmp directory of emmc
What have i do wrong?.
Thanks.
Click to expand...
Click to collapse
no log, no kernel info, no recovery info. cant help you.
Lanchon said:
no log, no kernel info, no recovery info. cant help you.
Click to expand...
Click to collapse
Hi.
Forgive me for not giving more information to solve the problem.
Yesterday I could partition the memory.
I had to change the partition type of memory EMMC to vfat and the script to start.
I used the IsoRec TWRP 2.8.7.0.
I used the last script publicated "lanchon-repit-20160415-system=1.0-data=same-sdcard=max-preload=min+wipe-i9100.zip"
Thanks
chapito said:
Hi.
Forgive me for not giving more information to solve the problem.
Yesterday I could partition the memory.
I had to change the partition type of memory EMMC to vfat and the script to start.
I used the IsoRec TWRP 2.8.7.0.
I used the last script publicated "lanchon-repit-20160415-system=1.0-data=same-sdcard=max-preload=min+wipe-i9100.zip"
Thanks
Click to expand...
Click to collapse
you didn't need to change your sdcard to FAT !!!
you could have used: -sdcard=max++ext4
if only you had read the instructions! that's what they are there for lol
well, it's your data you had to format, not mine!
thanks!