Database Info

[MOD]Auto Swap Extention

What you need:
1. Swap Capible ROM with Root Access (see list below)
2. A third Partition on your SD card, known as Linux-swap set between 16-32MB (Easiest way to achieve that for free if you don't have linux with gparted is to download the Ubuntu ISO disk and install it on your system, all for free. can even be installed through Windows or run straight off the disk. you can also follow the link Here.)
3. Research
4. Terminal or ADB
5. Thumbs
What to do:
1. download the attached file and copy it to your sdcard, unmount sdcard from computer.
2. in terminal or adb shell type:
don't type things in ().
Code:
$su (terminal only)
#mv /sdcard/userinit.sh /system/sd/
#chmod 775 /system/sd/userinit.sh
#sh /system/sd/userinit.sh
This should do it, no reboot required. if you want to verify that you have your swap space running now or after a reboot simply type:
Code:
#free
you should see total memory to the right of "Swap" be filled in with the number of bytes you set.
A little bit about Swap on the G1:
Swap partitions are currently completely seperate from ROMs. all ROMs should be able to do this, however I'm not specfically certain and don't have the time or patience to try in or do research for it.
YOU DO NOT NEED TO FLASH YOUR ROM AT THE SAME TIME TO GET THIS TO WORK. In fact, you will have to do this everytime AFTER you do a new flash or wipe. I'm getting mixed reports about this, I myself didn't have to redo this after flashing CM 3.5.3, however it is possible that some are not mounting /system before pushing/copying the file over, which makes thier effort pointless and explains the confusion!
Other Notes:
This script was written by Dwang, I had no part in it, I'm just trying to make it easier and seperate it from Cyanogens mod threads.
Also please understand that Linux does ./sdcard/ and /sdcard/ in linux, nor is there any difference from /sdcard/tmp/ and /tmp/ when you prompt to /sdcard/ (ie, typing cd /sdcard or cd sdcard)
Also, please do not set up your linux swap over 32MB, you're asking for trouble.
Swap Capible ROMs (Dev's PM me if you incorperate this into your ROM):
Cyanogen's MOD 3.5.2 + higher
JACHero w/ http://forum.xda-developers.com/showpost.php?p=4054111&postcount=19

Thank you.. will give this a try and let you know.

A small .bat file would be nice for this since it has to be done after every flashing
We might need a .bat file for all the .bat files i'm collecting.
Wasnt this experimental in CM's rom and it ran "sluggish"?

Unsure, and it runs fine. as far as a .bat.... why? it's all .sh. does .bat even work in linux?

Denkai said:
What you need:
YOU DO NOT NEED TO FLASH YOUR ROM AT THE SAME TIME TO GET THIS TO WORK. In fact, you will have to do this everytime AFTER you do a new flash or wipe.
Click to expand...
Click to collapse
I thought userinit.sh on /system/sd would survive a flash an/or wipe?
Also you might add that mkswap should be run once on a newly created swap partition.

looking over the userinit.sh...........
What exactly does this, .sh, do??

bkmo said:
I thought userinit.sh on /system/sdcard would survive a flash an/or wipe?
Click to expand...
Click to collapse
/system/sdcard doesn't exist. it's /system/sd and no, it doesn't it gets copied over.

Mikey1022 said:
looking over the userinit.sh...........
What exactly does this, .sh, do??
Click to expand...
Click to collapse
first part checks to find the 3rd partition and sets it up as linux swap. the second script checks for media files meant for ring tones and seperates them so that the file doesn't show up twice in your music player.

Denkai said:
/system/sdcard doesn't exist. it's /system/sd and no, it doesn't it gets copied over.
Click to expand...
Click to collapse
Sorry system/sd ...I was editing the post to correct it when you replied. I re-flashed cyanogen 3.5.2 without a wipe and userinit survived.

Denkai said:
/system/sdcard doesn't exist. it's /system/sd and no, it doesn't it gets copied over.
Click to expand...
Click to collapse
/system/sd doesn't get copied over on a flash - that's why userinit.sh is placed there, so users can put custom commands in a location where the ROM (just CM, AFAIK) will know to execute them.
If you aren't running CM 3.5.2 or greater you'll need to set this up somehow so that it is run on boot. Cyanogen has a line added to his A2SD script that will do that.
Now.. Any advice on how to create my swap partition if I don't have a memory card reader to use on a PC w/ Ubuntu? I tried a gparted live cd on my GF's Thinkpad, but the card reader wasn't detected. I'm sure there's some way to do it at the command line, but my linux-fu is out of practice and I never did mess with partitions much.
BTW, thanks for posting - I saw this in the CM experimental thread but kept forgetting to install, until I saw this post. Doing it now

Saiboogu said:
Now.. Any advice on how to create my swap partition if I don't have a memory card reader to use on a PC w/ Ubuntu? I tried a gparted live cd on my GF's Thinkpad, but the card reader wasn't detected. I'm sure there's some way to do it at the command line, but my linux-fu is out of practice and I never did mess with partitions much.
Click to expand...
Click to collapse
Gparted was not detecting my SD reader on my Dell until I installed the newest Gparted from source on Ubunty Jaunty.

bkmo said:
Gparted was not detecting my SD reader on my Dell until I installed the newest Gparted from source on Ubunty Jaunty.
Click to expand...
Click to collapse
OK -- I should have tried a Jaunty disk anyway, I just got sidetracked by the gparted disk not working. I'll try the newer Gparted version with that. Thanks.

Thanks for making a separate thread. You should probably put the post I wrote on how to make a swap partition in your first post.
http://forum.xda-developers.com/showpost.php?p=4029519&postcount=145

Instead of running it again and again, I would prefer it to be added to runme.sh in boot.img... so that it will be run automatically on every boot...
I am trying it now...

it only gets run once on boot, I believe. will add that post, thanks.

what level swappiness is everyone finding optimal?
i'm on a non class 6 microsd and if i set swappiness over 30, it doesn't take long for the system to get bogged down by IO

alapapa said:
what level swappiness is everyone finding optimal?
i'm on a non class 6 microsd and if i set swappiness over 30, it doesn't take long for the system to get bogged down by IO
Click to expand...
Click to collapse
yeah I think 30 may be a bit too high. I'm using 10 or 20 now.
Try 100 It actually makes the phone super responsive at first, but then it starts getting very laggy after a while.

sangeet.003 said:
Instead of running it again and again, I would prefer it to be added to runme.sh in boot.img... so that it will be run automatically on every boot...
I am trying it now...
Click to expand...
Click to collapse
The script was written for cyanogen's ROMs 3.5.2 or greater.
Those ROMs will automatically execute /system/sd/userinit.sh on bootup. Which means no modifying anything in update.zip

The attached boot image is for JACHERO 2.~r6 I have added the script to the runme.sh to mount swap on every boot i am trying to add the .29 kernel which has multitouch....
The swappiness is set to 80 which I feel Works Great Means +20 than the system Default(60) swappiness...
I have not tested it Coz i cant Partition the Sdcard currently with 1 more partition, Will be testing it later say in 5-6 hours...
Testers are appreciated...
Just extract it on computer & fastboot flash it.... on the boot partition...
Noobs Dont Try It

Saiboogu said:
/system/sd doesn't get copied over on a flash - that's why userinit.sh is placed there, so users can put custom commands in a location where the ROM (just CM, AFAIK) will know to execute them.
If you aren't running CM 3.5.2 or greater you'll need to set this up somehow so that it is run on boot. Cyanogen has a line added to his A2SD script that will do that.
Now.. Any advice on how to create my swap partition if I don't have a memory card reader to use on a PC w/ Ubuntu? I tried a gparted live cd on my GF's Thinkpad, but the card reader wasn't detected. I'm sure there's some way to do it at the command line, but my linux-fu is out of practice and I never did mess with partitions much.
BTW, thanks for posting - I saw this in the CM experimental thread but kept forgetting to install, until I saw this post. Doing it now
Click to expand...
Click to collapse
If you mount the sdcard from the menu bar (USB), it will also mount the ext partition too. From there u should b able to partition from within Ubuntu.

[UTIL] sdparted v0.6 - easy sdcard partitioning, upgrading to ext3/4

this script automates the process of partitioning a sdcard on your android device. it should work fine for all sizes/types of sdcards, but since i can't test all sizes/types of sdcards, we'll have to see.
if you are running into problems with the script, post the log file(located at /data/sdparted.log) when asking for help.
big ups to cyanogen (parted and upgrade_fs) and Denkai (upgrading to ext4).
i welcome all comments, questions & suggestions, related to the script. this is NOT a general q&a.
read the ENTIRE post BEFORE asking questions, please.
to manually partition your sdcard see my other thread.
DISCLAIMER: i take no responsibility for what happens to you, your phone, sdcard, data, sanity, etc if you use this script. remember to backup your sdcard to your pc before you begin. this script has the potential to COMPLETELY WIPE your sdcard!
##########################
shameless promotion:
sdparted can also be found in amon_ra's recovery and natalic's android toolkit.​
##########################
features:
-automated partition of sdcard using parted
-upgrading to ext3/ext4
-downgrading to ext2
-interactive mode
-supports units (M and G)
-supports floating point partition sizes (ie. .5G=512M)
-automatic logging to /data/sdparted.log​
##########################
requirements:
android phone with proper utilities(cm-recovery-1.4, amon_ra's recovery)
sdcard <--class 6 recommended(adata makes good ones...they max out the g1 hw in terms of speed)
adb
fingies​
##########################
Code:
sdparted v0.6 created by 51dusty
if you use this script in your work, please give some credit. thanks.
requirements: cm-recovery-v1.4
usage: sdparted [options]
options:
--fatsize|-fs SIZE[MG] set the size of the fat32 partition to <SIZE>.
default=total sdcard size - (ext + swap)
--extsize|-es SIZE[MG] set the size of the ext partition to <SIZE>.
default=512M
--swapsize|-ss SIZE[MG] set the size of the swap partition to <SIZE>.
if set to 0, no swap partition will be created.
default=32M
--extfs|-efs TYPE set the filesystem of ext partition to <TYPE>.
valid types=ext2, ext3, ext4
default=ext2
--upgradefs|-ufs TYPE upgrades existing ext partition to <TYPE>.
this operation will NOT wipe your sdcard and
cannot be used with any partition creation options.
valid types=ext3, ext4
--downgradefs|-dfs TYPE downgrades existing ext partition to <TYPE>.
this operation will NOT wipe your sdcard and
cannot be used with any partition creation options.
valid types=ext2
--interactive|-i interactive mode
--help|-h display this help
--printonly|-po display sdcard information
--silent|-s do not prompt user, not even initial warning.
examples:
sdparted creates swap=32M ext2=512M fat32=remaining free space
sdparted -efs ext4 creates swap=32M ext4=512M fat32=remaining free space
sdparted -fs 1.5G -efs ext3 creates swap=32M ext3=512M fat32=1536M
sdparted -es 256M -ss 0 creates no swap ext2=256M fat32=remaining free space
sdparted -ufs ext4 upgrades ext partition to ext4
##########################
need to backup your ext partition?
the following commands will backup/restore your ext partition to/from a folder named sdbackup in your current directory. these must be run while phone is booted, not recovery.
to backup your ext partition: adb pull /system/sd/ %CD%\sdbackup
to restore back to sdcard: adb push %CD%\sdbackup /system/sd
##########################
to run from computer:
1. download sdparted.txt below to computer
2. connect g1 via usb
3. boot into cm-recovery-v1.4, goto console(alt-x)
4. at your windows cmd prompt type: adb push /path-to/sdparted.txt /sbin/sdparted
5. then type: adb shell chmod 755 /sbin/sdparted
6. to run type adb shell and hit enter.
7. you can now run script (ie. sdparted -efs ext4).​
to run w/o computer:
1. download sdparted.txt below to root of sdcard. (get downloadcrutch if needed*thnx lycoln)
2. boot into cm-recovery-v1.4, goto console(alt-x)
3. at # prompt type: mount /sdcard
4. then, mv /sdcard/sdparted.txt /sbin/sdparted
5. then, chmod 755 /sbin/sdparted
6. you can now run script (ie. sdparted -efs ext4).​
you CANNOT run this script from terminal app.
##########################
if the script crashes or you receive an error:
immediately pull the log to your computer(using adb pull /data/sdparted.log sdparted.log), b/c the log will not be there after a reboot. send me the log when reporting errors, please.
##########################
changelog:
changes in v0.6
*new feature=interactive mode
*tweak user abort function for those with itchy fingers
changes in v0.5.3
*remove initial warning(kinda pointless if there is another before you wipe)
*clean partition table handling code
*general code cleanup/consolidation in anticipation of new features
changes in v0.5.2
*handling of "partition 1 may not be aligned to cylinder boundaries", fixing "sh: -gt: argument expected" and related phenomena (ie. sdparted trying to partition using negative numbers ) reported by midtoad
changes in v0.5.1
*slight change to logging(so slight it only gets a .1), fixing "/sbin/sdparted: line 5: .//sbin/sdparted: not found"
changes in v0.5
*new feature=floating point partition sizes
*new feature=downgrade ext filesystem (ext3->ext2 ONLY, for now)
*fix some crappy programming
changes in v0.4
*unmount all partitions before operations, fixing "Error: Partition(s) on /dev/block/mmcblk0 are being used".
*remove some useless code
changes in v0.3
*new feature=logging
*new feature=units
*new feature=print card info
changes in v0.2
*add cm-r-v1.4 check to prevent running in 1.3.1
##########################
todo items:
-downgradefs support for ext4
##########################

Excellent!
One of the easiest things Ive done in a while. Worked great!

great.
i'm new to android, so i have a question.
I'm guessing when you go to settings and eject SD card, it only unmounts the FAT32 partition, right?
So does that mean the only safe way to eject the card is shutting down?
great job!

SyXbiT said:
great.
i'm new to android, so i have a question.
I'm guessing when you go to settings and eject SD card, it only unmounts the FAT32 partition, right?
So does that mean the only safe way to eject the card is shutting down?
great job!
Click to expand...
Click to collapse
That depends which ROM you are running and how it's set up. CyanogenMod for example, will automatically use the Ext partition for Apps-to-SD if it finds one. Ejecting a card while the phone is using it as part of its internal memory would be a Bad Thing™.
If you are running the stock firmware, it would be safe to remove the card after ejecting it in settings.

Could anyone at their convenience post directions for using this with console,
and where to place text attached? Funny, I've done this the manual way but don't know the simple things.
Thanks in advance for your work and patience. =)

Figured this one out!

Can you tell me how exactly to run this script? I ran this in recovery and i get sdparted not found.

sacredsoul said:
Can you tell me how exactly to run this script? I ran this in recovery and i get sdparted not found.
Click to expand...
Click to collapse
where did you place the file? did you run chmod?

excellent! Great job dusty

sacredsoul, Make sure you have the latest Cyanogen's Recovery Image, which I used 1.4. I got the same error using 1.31. I then updated to 1.4 and it worked perfectly. great Post 51dusty.

i am tryin to do this...hw do i get in recovery mode and wat exactly im i typin?..i hav a 4gig

Just wanted to chime in and say this script worked beautifully for me on a 16gb card.
Thanks!

I don't get it, am I mounting first, placing the file on the SD, then rebooting into recovery? The instructions make it sound as if I can just boot into console and pull it off my machine. Many people might find that confusing.

ctheory83 said:
I don't get it, am I mounting first, placing the file on the SD, then rebooting into recovery?
Click to expand...
Click to collapse
...i don't say to mount anything.
ctheory83 said:
The instructions make it sound as if I can just boot into console and pull it off my machine.
Click to expand...
Click to collapse
actually, you can...but you push instead.
51dusty said:
to install from computer, download, boot into cm-recovery-v1.4, goto console, and adb push /path-to/sdparted.txt /sbin/sdparted, then adb shell chmod 755 /sbin/sdparted. you can now run the script from adb(in recovery) or from g1 recovery console.
Click to expand...
Click to collapse
i will edit post to make installation less "confusing"...done.

Hey dusty i been struggling with this for like a week now and finally decided to post! so i got the adb thing running on my pc, windows 32-bit and it finds my device when i put the command "adb devices", so that far im fine but then your instructions tell us to boot the phone into recovery console and the type "adb shell" the result comes up "no adb found", at this point i have tried it with my phone plugged into the pc and unplugged without mounting it from the G1... please help what am i doing wrong im really confused! thanks!

The script is great i finally figured it out how to work this thing, but i just a bit of an issue now, when i keep installing apps my internal storage is also getting reduced a bit, i have installed about 100mb of apps and my internal storage reduced to 67 from 70mb, i have already cleared cache, using the app to move it to sd card, and also turned my phone off and took out the battery and rebooted, but its standing at 67mb... does this possibly have anything to do with protected apps??? please help!!

adb not found
hi,
it says adb not found, what am i doing wrong ?.

so my problem is that im not quite sure on what you mean by the path in the following command
adb push /path-to/sdparted.txt /sbin/sdparted
i know that you mean where ever the file is located but, for instance i just put mine in the c drive, so it should be adb push /c drive/sdparted.txt /sbin/sdparted
what is the correct way that should look?
edit: ok so i found my problem, i tried to install it from the adb shell, that was wrong, and this is the string in windows cmd i used for sdparted located just on the c drive
"adb push %cd%\sdparted.txt /sbin/sdparted" it then says the speed it was written, etc. does that mean it is correctly installed? from here, im not too sure how to go about setting up my partitions, again im very new to this.

you can now run the script from adb(in recovery) or from g1 recovery console.
Click to expand...
Click to collapse
Everything went okay, only the last stap with running from adb i dont understand.
What command is that ?

CoopZor said:
Everything went okay, only the last stap with running from adb i dont understand.
What command is that ?
Click to expand...
Click to collapse
i have edited first post to clarify how to run.
to run, from adb:
1. boot into cm-recovery-v1.4, goto console(alt-x)
2. at windows prompt, type adb shell and hit enter.
3. once connected via adb, you can now run script (ie. sdparted -efs ext4).
Click to expand...
Click to collapse

[Think-Tank] Bad Blocks causing missing apps & broken functions

Seems that a refurbished G1 I received has 3 blocks stuck and marked in eccUnfixed for some time, pulled the info from command:
Code:
cat /proc/yaffs
Issues:
I'm thinking as our G1s are getting older and with ROM flashing abuse we are killing NAND blocks on them fast. Researching at various sites linux-mtd & Yaffs.net, the garbage collector of Yaffs should be marking and removing these if Yaffs was controlling the ECC mode. However since Yaffs G1 android setting relies on NANDecc for marking bad blocks I'm not so sure which is cleaning up and not retiring these blocks, maybe NAND and YAFFS are in disagreements or one has ability to retire these blocks and end this problem. It seems though that while flashing ROMs data is being written on to these areas and after a remount it is evident that something may be wrong.
Experiences:
An app could go missing or a file or folder name with invalid characters may happen or a phones feature may suddenly stop workin such as scanning for wireless networks timing out.
The issue is reproducable on freshly erased partitions via fasboot erase system -w.
If a dev would like to co-ordinate with me I will grant them remote access to a system with said phone connected via usb with adb port redirection/remote adb or desktop access with adb and fastboot and vnc client with vnc server on phone .
Workaround:
Working with xda member Licknuts we tested numerous times with a few ROMs 1.6 and 2.1 and came up with this for a quick work around that basically involves erasing /system via fastboot then in recovery lock /system from being formatted and unmounted after ROM Flashing you then tarball /system partition to /sdcard do a reboot into fastboot and erase system again reboot into recovery and untarball back to /system.
Prerequisite:
A phone that suffers missing apps in /system or odd phone behavior features after fresh ROM flash.
ADB with Fastboot, USB Cable connected to your Phone.
Adapt the procedure bellow for issues stemming from the /data partition.
<Go for it>
Going to have to clear out system as it probably has corruption.
Code:
fasboot erase system
Note:::
Hboot spl dictates reboot & some engineer spls have no menu option and engineer spls do not reboot to recovery with an empty partitions when USB cable is connected so disconnect it and reboot then reconnect when in recovery console or just fastboot flash system system.img using HTC 1.6 system.img extracted from their zip.
FYI:::
Haykura sapphire spls (aka: danger spl / death spl) are able to boot to recovery with only radio/hboot/recovery partitions intact as long as it is instructed to do so other wise you get G1 screen with red[fastboot] with USB cable attached (might be the missing safety mode for bricked G1s but not certain).
Anyhow just Reboot to recovery console from menu option or [send][menu][hang] quickly hold [home]
Using adb shell we can lock system partition from being formatted and unmounted by ROM Flash scripts instructions in scripts.
Code:
adb shell
mount -a
cd /system
flash ROM and any updates from Recovery console on phone as normal...... and then we tarball the /system partition before it remounts on reboot and corruption happens
While still in adb shell and hanging out in /system, issue these commands to mount the sdcard and create a tarball of /system.
Code:
mount /sdcard
tar -cvf /sdcard/system.tar /system
sync
reboot bootloader
in fastboot we erase system again because most likely data will corrupt because the Flashing procedure may have written to bad blocks and when mounted or remounted it will corrupt, this is why we tarballed /system before it happens.
Code:
fastboot erase system
then reboot back to recovery console
using adb shell we un-tarball system.tar back to /system mountpoint with YAFFS that supposedly knows about these bad blocks so writing filles out these should be checked and avoided.
Issue these commands:
Code:
mount /sdcard
mount /system
tar -xvf /sdcard/system.tar -C /
sync
reboot
reboot as normally would and with some luck files wrote out avoiding those marked bad blocks.
Resolutions and notes:
flash_image and ROM flashing scripts may be flashing directly to device level which does not have YAFFS bad block information and once mounted with a file system, corruption happens from file system having bad block information.
Using sapphire spls (danger spl) we have access to fasboot oem task 28 & fastboot oem task 29 will low level format, repair, report and mark blocks bad but doesn't seem to resolve issues maybe YAFFS doesn't sync or care about the blocks marked in NAND mtd.
Possibly we need a compiled kernel and complete compiled tool set to fully work on Nand device mtd and format tools for Yaffs and scripts to automate some tasks. Seems Nandroid and /system/xbin, /sbin, /system/bin and recovery console tool-sets are limited as well.
*I think and type fast so note that typos may exists hopefully not in coding but elsewhere and if you exist solely to point out grammar and spelling of others on internet forums or broken English of members from other countries, then respect my preemptive "go **** yourself".
If my help aided in getting your phone in working order then give a family member or friend a call or donate some $ to the devs that keep our phones fun to use.

Have you got any more in-depth info on this?
The way I see it, you erase with fastboot... What does this accomplish that doesnt happen if one from the recovery console or adb, dd /dev/zero into the MTD device?
Or, better yet, wouldn't tarring the system partition after its written just copy the bad files that existed on the bad erase block(s)?
Or am i seriously not understanding what your instructions are doing.. To me:
Erase in some way once. What this does, i'm unsure.
Flash
Mount /system and tar to sdcard
Erase again (again what this does to assist the process is uncertain to me)
Boot to recovery
Mount everything
Untar from sdcard.
Tell me if I understand this wrong?

as some blocks are reserved to replace bad blocks , most of the time you won't see a problem
However, once these are used up you will effectively see partition 'shrinkage'
we can limit this 'shrinkage by using lbcoder's custom MTD via the kernel command line
I have a relatively simple way of implementing this
http://forum.xda-developers.com/showthread.php?p=7061471#post7061471

Thanks Phonekenstein for such a great input. I never knew this thread was exist until now but I found this and read the title, damn this is exactly what my case is!
Follow your instruction and re-install the rom, every single headached weird behaviors are disappear and the system runs like butter. It seems my phone is finally fixed!
I think this thread should get more attention.

(bump)
This is a great find! I will be trying this on my HD2 as I have some bad blocks also causing Android builds to go wack.

[Tool][TWRP][ORS]Simplified backup/restore and flashing

What this is: it's a shell script that generates commands for TWRP using its Open Recovery Script (ORS) language. See: https://twrp.me/faq/openrecoveryscript.html
Over the years I've found that I repeatedly did the same set of actions in TWRP, with just file names changing. Being inherently lazy I decided to automate these processes. So now I simply do the following in a Terminal session on my phone:
To backup my current rom: . bf
To backup my current rom and update my current rom (flash over the current rom) which is sort of like an OTA update: .bf -o
To NOT backup my current rom, restore a previous backup, and clean update the rom (wipe system, flash updated rom, Gapps, and Magisk): .bf -brf
To do a backup and clean install (wipe data and system) a rom: . bf -c
The script generates the ORS commands in /cache/recovery/openrecoveryscript and reboots the phone into recovery. TWRP will automatically process the file and, if there are no errors, reboot the system.
Requirements:
- the script must run as root (for access to /cache)
- flashed files must all be in a single directory (/sdcard/@install) and adhere to the following naming conventions:
File names (no spaces):
rom-*.zip
gapps-*.zip
kernel-*.zip (if flashed after Magisk)
Kernel-*.zip (if flashed before Magisk)
MyMods-*.zip (regular mod)
MyTemp-*.zip (one-off mod/fix)
su-*.zip (Magisk or other root installer)
Note: there must only be one file of each these types in the folder, if required. I normally prefix files that I'm not using with an x to avoid multiples, for example I'll have rom-lineage-17.1-20200223* and xrom-lineage-17.1-20200205* in the directory
- a system property (my.version) must be set as it's used to name new backups. I used to add the property to build.prop (using a MyMods zip) but now I use a Magisk MyMods module.
- two additional scripts (depending on the options chosen) are needed to be called in TWRP and must exist in /sdcard/Tools. They are:
rmpwd (remove the lock screen password after restoring a backup)
wipesys (wipes /system or /system_root because TWRP lost the 'wipe system' command - but not 'wipe data' - in version 3)
Limitations:
I've only used it on A-only systems. I have no idea if it works on A/B.
Design notes:
- this is not an executable script because files in /sdcard can't be. But I wanted it (and all my other scripts) in /sdcard so that it is easily accessible. That's why there are no exit statements (only returns) in it as it's run in-line using ". bf" or "source bf".
Usage notes:
- I recommend adding the directory to $PATH (export PATH=$PATH:/sdcard/Tools)
- many of the options (for example b,p,m,s,...) are toggles (on/off switches). They can easily be "flipped" by adding them to the command options. For example, -b is on by default, so specifying it again (. bf -b) turns it off (. bf -bb would leave it on).
- the -v option caters for the transition from one Android version to another. It makes it possible to have a sub-directory with the new version files (gapps, rom, mods), yet using the same su version. For example, if I'm currently on Android 9 but want to test Android 10, then I'd create a directory under /sdcard/@install called 10 (or Q or whatever). then I could switch from my current rom to a new 10 rom (clean install) on using .bf -cv 10 and the script will look in /sdcard/@install/10 for the gapps, mods, and rom. Always specify the -v option last in the list or separately (. bf -...v 10 or . bf -... -v 10).

Reserved.
Sent from my OnePlus 3T using XDA Labs

Thank you very much for sharing your scripts and ideas. I did not even know that there is a chance to script TWRP . My motivation to start my Android Backup and Restore Tools project were very similar to yours ... beside as I am on a Pixel 3a with A/B-system and not having TWRP available (yet(?)) I was looking for a way to do the backup and restore process without TWRP at all.
In addition I prefer transferring the data directly to a remote device. The storage on the devices is always limited and having a backup on the device itself (even if it is only for a limited time period) may cause trouble or at least limits the possible size of the backup which can be done.
Ideal for me would be a method to extract the important components of a ROM (/system, /vendor, /data, ...) off of the device and package them later in a way that they can be flashed back to the device later on e.g. using fastboot. As the new devices go into the direction of userspace fastboot (fastbootd) I am wondering if there would we the chance to "enhance" the fastbootd mode in a way that it can be used as a kind of recovery mode, too.
I am going off topic and just sharing my dreams only ... nevertheless thank you again for sharing!

AndDiSa said:
Thank you very much for sharing your scripts and ideas. I did not even know that there is a chance to script TWRP . My motivation to start my Android Backup and Restore Tools project were very similar to yours ... beside as I am on a Pixel 3a with A/B-system and not having TWRP available (yet(?)) I was looking for a way to do the backup and restore process without TWRP at all.
In addition I prefer transferring the data directly to a remote device. The storage on the devices is always limited and having a backup on the device itself (even if it is only for a limited time period) may cause trouble or at least limits the possible size of the backup which can be done.
Ideal for me would be a method to extract the important components of a ROM (/system, /vendor, /data, ...) off of the device and package them later in a way that they can be flashed back to the device later on e.g. using fastboot. As the new devices go into the direction of userspace fastboot (fastbootd) I am wondering if there would we the chance to "enhance" the fastbootd mode in a way that it can be used as a kind of recovery mode, too.
I am going off topic and just sharing my dreams only ... nevertheless thank you again for sharing!
Click to expand...
Click to collapse
Thanks for your post. I find it interesting how different requirements produce very different solutions.
One of my main drivers is to do everything on the device itself and avoid using a PC if at all possible.
Sent from my OnePlus 3T using XDA Labs

BillGoss said:
What this is: it's a shell script that generates commands for TWRP using its Open Recovery Script (ORS) language. See: https://twrp.me/faq/openrecoveryscript.html
Click to expand...
Click to collapse
Interesting project, thanks.. TWRP on my a/b device (OP7) doesn't show options to backup system or vendor partitions, just the images. Do you happen to know if ORS supports image backup?

Tirofog said:
Interesting project, thanks.. TWRP on my a/b device (OP7) doesn't show options to backup system or vendor partitions, just the images. Do you happen to know if ORS supports image backup?
Click to expand...
Click to collapse
Unfortunately ORS doesn't do images. See the documentation at https://twrp.me/faq/openrecoveryscript.html
Sent from my OnePlus 3T using XDA Labs

@BillGoss Hello, your script shows that it must be possible what I am looking for although that is quite different from what your script provides.
I have two rooted phones with LineageOS based ROMs that don't fully shutdown. If I shut them down from the system they drain the battery more than while in use. Therefore I must first reboot them to recovery and from there shut them down.
I am looking for a script that does exactly that: after being started in system mode reboot to recovery and then poweroff.
I have little knowledge in scripting and thus don't understand your script to reduce it to the requested commands.
Could you, please, help me with such a script?

bege10 said:
@BillGoss Hello, your script shows that it must be possible what I am looking for although that is quite different from what your script provides.
I have two rooted phones with LineageOS based ROMs that don't fully shutdown. If I shut them down from the system they drain the battery more than while in use. Therefore I must first reboot them to recovery and from there shut them down.
I am looking for a script that does exactly that: after being started in system mode reboot to recovery and then poweroff.
I have little knowledge in scripting and thus don't understand your script to reduce it to the requested commands.
Could you, please, help me with such a script?
Click to expand...
Click to collapse
Have you tried just running /system/bin/reboot -p in a Terminal session? That should shutdown your phone.
To do it through TWRP:
Create a file called openrecoveryscript containing the following command cmd reboot -p
Copy the file to /cache/recovery and reboot into recovery.
You can look at the finalize function in my script for how to create your own script to do this.

BillGoss said:
Have you tried just running /system/bin/reboot -p in a Terminal session? That should shutdown your phone.
To do it through TWRP:
Create a file called openrecoveryscript containing the following command cmd reboot -p
Copy the file to /cache/recovery and reboot into recovery.
You can look at the finalize function in my script for how to create your own script to do this.
Click to expand...
Click to collapse
Thank you for your help.
I tried everything to solve the incomplete poweroff issue, but on one phone it is a known issue in the LineageOS ROM, on the other phone it is an unsolved issue related to Magisk.
Poweroff in TWRP with a openrecoveryscript works without cmd in your answer.
When the phone reboots to TWRP I must decrypt the data partition before the script runs. This is not necessary to just poweroff the phone and an OTA update starts running right away.
What does an OTA update do differently when it reboots to TWRP? What must I do to skip the password screen in TWRP when an openrecoveryscript is in /cache/recovery ?

An ota doesn't use a recovery script. It uses a different way of invoking TWRP. I never liked into it.
You'll need to see if a Los developer can explain to you exactly how an ota works.

BillGoss said:
An ota doesn't use a recovery script. It uses a different way of invoking TWRP. I never liked into it.
You'll need to see if a Los developer can explain to you exactly how an ota works.
Click to expand...
Click to collapse
Meanwhile I noticed that this behavior is only on my A/B device where the OTA update is not installed by a TWRP script but before rebooting already. The other, older device is no A/B device and there TWRP starts with the openrecoveryscript right away. But that device is not encrypted. So I am not sure whether it is the standard behavior or due to the fact that is not encrypted.
Do you use your script on an A/B device? Is your phone encrypted? How does TWRP behave?

bege10 said:
Meanwhile I noticed that this behavior is only on my A/B device where the OTA update is not installed by a TWRP script but before rebooting already. The other, older device is no A/B device and there TWRP starts with the openrecoveryscript right away. But that device is not encrypted. So I am not sure whether it is the standard behavior or due to the fact that is not encrypted.
Do you use your script on an A/B device? Is your phone encrypted? How does TWRP behave?
Click to expand...
Click to collapse
Yes, I use my script on my A/B phone. I always run my phone's encrypted. And the behaviour of TWRP is the same as on A-only.
But note that I never used the Los ota updates. I've always downloaded the updates manually then used my script to backup and install.
I don't use TWRP for ota updates on my A/B phone because they are designed to run on a running system.

BillGoss said:
Yes, I use my script on my A/B phone. I always run my phone's encrypted. And the behaviour of TWRP is the same as on A-only.
But note that I never used the Los ota updates. I've always downloaded the updates manually then used my script to backup and install.
I don't use TWRP for ota updates on my A/B phone because they are designed to run on a running system.
Click to expand...
Click to collapse
Do you need to enter your password/pattern before your script runs in TWRP? Even if you e.g. only run a system backup that does not need decrypted data?

Well I suppose decryption is needed as soon as TWRP wants to mount an encrypted file system. So if e.g. the /etc/fstab of the initial ramdisk (which is used by the kernel when booting TWRP) contains a mount point with a an encypted filesystem TWRP will automatically ask for the key ...

bege10 said:
Do you need to enter your password/pattern before your script runs in TWRP? Even if you e.g. only run a system backup that does not need decrypted data?
Click to expand...
Click to collapse
Yes, I do: the backup is written to internal storage which is encrypted. Also, /cache is actually /data/cache on my A/B phone so the storage needs to be decrypted for TWRP to read the ORS.

BillGoss said:
Yes, I do: the backup is written to internal storage which is encrypted. Also, /cache is actually /data/cache on my A/B phone so the storage needs to be decrypted for TWRP to read the ORS.
Click to expand...
Click to collapse
OK, thank you, then I understand how it works and that I cannot bypass decryption for my workaround for the shutdown issues. I hoped to make it less laborious.

@BillGoss Hello, I once more need your help.
Inspired by your script I tried a simple openrecoveryscript to backup with TWRP.
When I put this line from your script (reduced)
Code:
backup BD `date +%y%m%d-%H%M%S`
into openrecoveryscript I get the error
Backup name: '`date' contains invalid characters: '`date'
Click to expand...
Click to collapse
(translated from German)
A similar problem occurs when I try to set variables: When the script is running it uses any text after the variable name literally as a string. Backticks, quotes, using "cmd" don't change anything.
And using a variable as backup name throws
Backup name: '$name' contains invalid characters: '$name'
Click to expand...
Click to collapse
In the TWRP terminal setting and using variables works as expected.
Does this still work for you? Do you have an idea why using the date command and variables in an ORS doesn't work here?
I use TWRP 3.6.0_9.0 on Fairphone 3, /e/OS (based on LineageOS for microG, Android 10)

bege10 said:
@BillGoss Hello, I once more need your help.
Inspired by your script I tried a simple openrecoveryscript to backup with TWRP.
When I put this line from your script (reduced)
Code:
backup BD `date +%y%m%d-%H%M%S`
into openrecoveryscript I get the error
(translated from German)
A similar problem occurs when I try to set variables: When the script is running it uses any text after the variable name literally as a string. Backticks, quotes, using "cmd" don't change anything.
And using a variable as backup name throws
In the TWRP terminal setting and using variables works as expected.
Does this still work for you? Do you have an idea why using the date command and variables doesn't work here?
I use TWRP 3.6.0_9.0 on Fairphone 3, /e/OS (based on LineageOS for microG, Android 10)
Click to expand...
Click to collapse
As you've discovered an ORS is not a Bash script and, therefore, you cannot do things like set variables or use inline commands with `...`. It is processed, I believe, by the "twrp" program inside TWRP.
You could, in the TWRP terminal, run
Code:
twrp backup BD `date +%y%m%d-%H%M%S`
If you want to run a shell command in ORS you base to write cmd <shell command> where <shell command> is the actual command.
But what you should be doing is running a shell script on your phone that produces an ORS file which doesn't need to access the shell environment. That's what my bf script is.

Thank you for your quick answer.
Oh no! I didn't realize that your bf script does not write the commands in backticks and the variables into the ORS but their values.
Thank you very much.

In case other ORS noobs like me come around and only want a simple script for regular backup and shutdown I post my solution here.
The bash script ORS_backup_shutdown.sh that creates the openrecoveryscript (this script is part of a Tasker task that afterwards copies and renames the file to /cache/recovery/openrecoveryscript and reboots to recovery):
Code:
scr=/sdcard/Documents/Z/ORS/ORS_backup_shutdown
echo > $scr # initialise ors
slot=`getprop ro.boot.slot_suffix` #active slot for A/B device
dat=`date +%F`
tim=`date +%H-%M-%S`
nam=$dat"--"$tim"_e_FP3-db"$slot # adapted from the default backup name of TWRP
echo set tw_storage_path external_sd >>$scr # internal SD = sdcard
echo backup bdmo $nam >>$scr # m = skip digest, o = compress
echo reboot poweroff >>$scr
And the openrecoveryscript outcome:
Code:
set tw_storage_path external_sd
backup bdmo 2022-05-09--09-18-02_e_FP3-db_b
reboot poweroff

[SOLVED] Help to mount userdata in linux or repair userdata.img

After an Update of Two magisk modules, my Ulefone Armor 11 5G staied stucked on boot logo, I can only enter in recovery or fastboot.
I try to build a TWRP, but it is not able to mount userdata.
I was able to download with the help of mtkclient all the partition on my phone, even userdata , it took 7 hours.
I wanted to load the image in linux but using mount disk imag or using the command sudo mount -o loop userdata.img ~/Armor_11_5G doesn't do anything not even an error message.
I'm wondering if the filesystem was corrupted during the update.
Is it possible to repair the fylesystem like in Windows?
Thanks

did you previously disable encryption and factory reset long time before the modules updates failed?
what do you mean mount doesn't do anything not even an error message? either it give error message or it succeed.

I didn't disable encryption before updating the modules, I already updated this modules many times.
what do you mean mount doesn't do anything not even an error message? either it give error message or it succeed.
Click to expand...
Click to collapse
That is the problem, it doesn't succed and I don't have an error message. The file is 256 Gbyte big, I don't know if it plays a role. I'm using Ubuntu 22

if phone is encrypted that's just 256G garbage. post the output of
Code:
$ parted <file> unit B print

Here are the results of parted
Code:
Error: /home/*****/Public/userdata.bin.img: unrecognised disk label
Model: (file)
Disk /home/osboxes/Public/userdata.bin.img: 249208733696B
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
Thanks

You may try https://www.cgsecurity.org/wiki/TestDisk_Download
I'd be glad for your feedback.

sorry thought it's whole disk, but it's only 232G file therefore parted won't print partition table
does apply to FDE only
assuming this file is dump of single userdata partition, open with HxD editor. if the partition image is not encrypted, you will see lot zeros within first 1024 bytes.
in that case you can check for file system type is ext4 or f2fs.
Code:
$ xxd -l 1080 dump.img | grep 53ef
$ xxd -l 1024 dump.img | grep 1020.f5f2
But most likely the userdata partition is encrypted, therefore no way to recover data offline.
The easiest way not to load Magisk modules is, not to load Magisk. Flashing stock boot.img will solve it.
Beware, in case you disabled encryption beforehand, booting stock boot.img will force encryption. This may take long time without notice.

I never disabled encryption, I don't know if Magisk do it without informing. I used this phone for an year without a problem.
After the update I left the phone on for one night but nothing happens. I tried to reflash the stock boot image, and again a whole night wait, but again nothing happens.
Reading the fstab the file system should be ext4.
The file is the dump of the whole userdata partition of my Ulefone Armor 11 5G.
I did a backup of the whole system before doing any experiment so if the partition table is corrupted maybe if I reflash back the userdata partition with a working partition table I have again access to the datas.

there is no partition table in userdata partition, I just gave you wrong advise. because the phones total storage is 256G, I made wrong assumptions (you can view partition table from file pgpt.bin)
full 1:1 backup is impossible for FBE encryption because encryption keys are stored in TEE. once you factory reset device backup of userdata + metadata becomes useless.
fstab doesn't tell you what file you just have dumped. if you can't find ext4 super magic (#7) it's impossible to loop mount that file (and impossible to decrypt on linux PC)
if you can't fix boot-loop by stock boot.img then it's unrelated to magisk modules. you can however enable adb in default.prop and capture adb logcat during boot-loop for further analysis. you could also inject own script that deletes some files (only DE encrypted files, CE encrypted files requires lock screen credentials aka pin/pattern)

boot this TWRP and get log from adb
Code:
$ fastboot boot recovery.img
$ adb shell twrp decrypt '1234'
$ adb pull /tmp/recovery.log
https://twrp.me/faq/openrecoveryscript.html

full 1:1 backup is impossible for FBE encryption because encryption keys are stored in TEE. once you factory reset device backup of userdata + metadata becomes useless.
Click to expand...
Click to collapse
With mtk client I was able to do the backup of tee1 and tee2 and also of gpt_backup and gpt_main.
$ xxd -l 1080 dump.img | grep 53ef
$ xxd -l 1024 dump.img | grep 1020.f5f2
Click to expand...
Click to collapse
Doesn't produce any results.
I have immediately the command prompt.
you can however enable adb in default.prop
Click to expand...
Click to collapse
How can I do that? Which value should I change in default.prop?

so your "backup" is encrypted. please note Trustonic Kinibi is TEE OS running in secure memory one can't access or backup with mtkclient. the tee partitions in phone storage do not contain any encryption key (none of the partitions does, secure memory is not even a partition). the only crypto related partition is metadata used for keydirectory of metadata encryption (on top of FBE encryption) but it is useless for backup purposes.
yes you can modify default.prop in boot.img, ro.secure=0 should give root access.
https://forum.xda-developers.com/t/...hone-with-broken-screen.2965462/post-85905033
Code:
ro.secure=0
ro.debuggable=1
persist.service.adb.enable=1
in case the default.prop modification is not sufficient, you need additional command to be executed as root.
Code:
# settings put global adb_enabled 1
as you installed magisk, you could use magisk overlay.d/sbin/ for running startup script.
https://forum.xda-developers.com/t/...ithout-losing-your-data.4383255/post-86934375
aIecxs said:
boot this TWRP and get log from adb
Code:
$ fastboot boot recovery.img
$ adb shell twrp decrypt '1234'
$ adb pull /tmp/recovery.log
https://twrp.me/faq/openrecoveryscript.html
Click to expand...
Click to collapse
How about this TWRP? it should be able to decrypt userdata. if decryption failed, provide recovery.log

Until tomorrow I cannot do a logcat and I cannot find my view logcat on my laptop.
I unpacked boot.img with Carliv Image Kitchen and there is no default.prop, that is present in the recovery as prop.default.
Is there a way to backup secure memory of Trustonic?
How about this TWRP? it should be able to decrypt userdata. if decryption failed, provide recovery.log
Click to expand...
Click to collapse
I already tried that version but it cannot decrypt, that' why I'm trying to build my own version of TWRp with the help of the creator of that version of TWRP, but I'm stucked.

oh, you know how to build TWRP with proper FBE + metadata encryption support? have a look at other Oppo devices how they did... good luck.
regarding default.prop in boot.img (it's a symlink to system unfortunately) you can do it the other way
use magisk overlay.d/sbin/
create a boot script that does the thing with resetprop -n <prop_name> <prop_value>
don't use outdated Carliv Image Kitchen! use osm0sis AIK from link above.

oh, you know how to build TWRP with proper FBE + metadata encryption support? have a look at other Oppo devices how they did... good luck.
Click to expand...
Click to collapse
I'm learning.
I try to integrate the decryption service following the suggestion of ADeadTrouser on Github, but the service doesn't want to start and I don't understand why.
I never checked Oppo, I will take a look at them also, thanks for the suggestion.

I think I figured out now the adb logcat at least. hope that helps
https://forum.xda-developers.com/t/accessing-my-phone-with-a-dead-screen.4542763/post-88016019

I tried your script butr nothing happens, the telephone is not listed when I type
Code:
adb devices
and if I type
Code:
adb logcat
I receive the message waiting for device

you might follow the thread

wenyendev said:
You may try https://www.cgsecurity.org/wiki/TestDisk_Download
I'd be glad for your feedback.
Click to expand...
Click to collapse
I run the software on the image and it identify the contents and can read the encrypted and not encrypted part, that means that all the files are there, but I cannot mount in Linux or in TWRP

The fact that I cannot mount in Linux or TWRP the userdata image/partition can be that is corrupted the partition or the file index?
That would also explain why the script for Magisk provided by aIecxs is not able to copy the adb_key from the cache in the data partition.