[Tool] LG Back up & restore (com.lge.bnr) .lbf file analysis & extract tool - Android General

Hello, recently I looked into LG backup and restore application using static analysis.
Note: this is based on version lg bnr 8.0.26 from 2020-02-13, tested with 7.0.20 from 2008-12-31
Here is the extract tool:
https://github.com/Mysak0CZ/LBFtool
And here is the technical part (you don't need to read this, if you just want to extract your data):
The file contains 2 headers (one of them is referred to as "new")
These are XML documents encrypted using AES/ECB/PKCS5Padding + SHA256 and null IV with static password
Code:
cd562f61-5399-3978-ac76-7c54b0508010
And they look like this:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<BackUp>
<Common>
<Name>LGBackup_200217_1</Name> <!-- Name of the backup file -->
<Id>Randomly generated UUID</Id> <!-- Id of the backup file -->
<IsNew>True</IsNew> <!-- IsNew is same in new and old header -->
<Version>1.00</Version>
<Attr Name="product_model">LM-X525</Attr>
<Attr Name="os_version">9</Attr>
<!-- And other attributes including google email -->
</Common>
<BackupItem Category="settings_DB" StartOffset="59392" DataSize="136626" />
<BackupItem Category="MEDIA_DB" StartOffset="901169" DataSize="78393344" FileName="external.db" />
<!-- Other backuped items -->
</BackUp>
The "new" header has more information than original header, but same structure.
The attribute "IsNew" signals the presence of the new header. (and is True in both files)
All numbers inside file are in Big-endian.
And this is the structure of the file (decimal position):
Code:
0: Unknown - 0x00 for me
1: Int32 - size of the new header
5: Int64 - offset to the new header (usually 13)
13 ... EOF-33: data
EOF-33: 20 bytes - SHA1 hash of the password (if present, otherwise still data)
EOF-13: Int64 - offset to the (old) header
EOF-5: Int32 - size of the (old) header
EOF-1: Bool - Lock byte - the file is (0x01) or isn't (0x00) protected by password
The password is only checked during import, not used to encrypt data (so to remove password, just set the last byte to 0)
Every "BackupItem" entry has a "Category".
Special case is Category="APPLICATION" with this structure:
Code:
<BackupItem Category="APPLICATION">
<APPLICATION Name="com.lge.ime" StartOffset="19221586" DataSize="2385983" Package="com.lge.ime">
<Attr Name="com.lge.ime">com.lge.ime</Attr>
</APPLICATION>
<APPLICATION Name="org.exmple.app" StartOffset="1429652009" DataSize="3420160" FileName="org.exmple.app.tar" Package="org.exmple.app">
<Attr Name="org.exmple.app">org.exmple.app</Attr>
</APPLICATION>
<!-- ... -->
</BackupItem>
All applications are tar files that contain 2 folders:
app - contains the apk file
data - application's internal data
All other items have:
StartOffset - Offset inside file
DataSize - Self explanatory
FileName - Not all have this, but most do; it is the original name of the backuped file.
The files are not compressed nor encrypted.
Footnote: This is my first post on XDA, so sorry for any mistakes I made (including my english) and feel free to correct me/give me advice.

Mysak0CZ said:
Hello, recently I looked into LG backup and restore application using static analysis.
Note: this is based on version lg bnr 8.0.26 from 2020-02-13, tested with 7.0.20 from 2008-12-31
Here is the extract tool:
And here is the technical part (you don't need to read this, if you just want to extract your data):
The file contains 2 headers (one of them is referred to as "new")
These are XML documents encrypted using AES/ECB/PKCS5Padding + SHA256 and null IV with static password
And they look like this:
The "new" header has more information than original header, but same structure.
The attribute "IsNew" signals the presence of the new header. (and is True in both files)
All numbers inside file are in Big-endian.
And this is the structure of the file (decimal position):
The password is only checked during import, not used to encrypt data (so to remove password, just set the last byte to 0)
Every "BackupItem" entry has a "Category".
Special case is Category="APPLICATION" with this structure:
All applications are tar files that contain 2 folders:
app - contains the apk file
data - application's internal data
All other items have:
StartOffset - Offset inside file
DataSize - Self explanatory
FileName - Not all have this, but most do; it is the original name of the backuped file.
The files are not compressed nor encrypted.
Footnote: This is my first post on XDA, so sorry for any mistakes I made (including my english) and feel free to correct me/give me advice.
Click to expand...
Click to collapse
Nice job dude! just a the quick recommendation: I would suggest that you put your code on gitlab or GitHub so it will become easier for others to contribute
Sent from my OnePlus 6T using XDA Labs

Hi. I'm trying to use this tool but I don't know how to use it. I followed the instructions and downloaded Python. but none of the other steps are working for me. I have my backup on my PC and am trying to extract my files into normal formats as I wish to move away from the LG platform. Thank you.
---------- Post added at 07:20 AM ---------- Previous post was at 06:57 AM ----------
Nevermind, I found a work around.
For anyone else in the future wondering: If you select backup with the app on the phone and backup to the internal memory, it will create .pdf files of all text/images in internal storage>>Quickmemo+. This doesn't help to retrieve the audio or video recordings, but at least that way you can retrieve your text files/images.

Tried script on old backup, failed with error.
Mysak0CZ said:
Hello, recently I looked into LG backup and restore application using static analysis.
Note: this is based on version lg bnr 8.0.26 from 2020-02-13, tested with 7.0.20 from 2008-12-31
Here is the extract tool:
And here is the technical part (you don't need to read this, if you just want to extract your data):
The file contains 2 headers (one of them is referred to as "new")
These are XML documents encrypted using AES/ECB/PKCS5Padding + SHA256 and null IV with static password
Code:
cd562f61-5399-3978-ac76-7c54b0508010
...
[/CODE]
All applications are tar files that contain 2 folders:
app - contains the apk file
data - application's internal data
All other items have:
StartOffset - Offset inside file
DataSize - Self explanatory
FileName - Not all have this, but most do; it is the original name of the backuped file.
The files are not compressed nor encrypted.
Footnote: This is my first post on XDA, so sorry for any mistakes I made (including my english) and feel free to correct me/give me advice.
Click to expand...
Click to collapse
My first post as well. Thanks for the script. I ran it on an old backup I was hoping to extract contacts from. See excerpt with error below.
C:\Temp\LG G4\Backup>python extract.py LGBackup_190218.lbf
[+] Backup size: 597864605
Parsing...
Reading header
[*] Header start offset: 597792768
[*] Header length: 71824
[+] Header read OK
Reading header (new)
[*] Header start offset: 7021235167904623918
Traceback (most recent call last):
File "extract.py", line 125, in <module>
root = readHeader(True)
File "extract.py", line 99, in readHeader
size = getBackupInfoSize(isNew)
File "extract.py", line 88, in getBackupInfoSize
raise Exception("Invalid header length.")
Exception: Invalid header length.
C:\Temp\LG G4\Backup>
Click to expand...
Click to collapse
I'm a complete newbie with Python, haven't done any coding in over a decade. Where do I go from here?

Ninja_Toad said:
My first post as well. Thanks for the script. I ran it on an old backup I was hoping to extract contacts from. See excerpt with error below.
I'm a complete newbie with Python, haven't done any coding in over a decade. Where do I go from here?
Click to expand...
Click to collapse
Hi! Your backup seem to be at least partially corrupted. (The new header position seems to start at 7021235167904623918, however the file is only 597864605 bytes long). Fortunatly there are two headers in the backup and the old one seems to be ok (You got "[+] Header read OK" for the old header). So you can try ignoring the "new" header.
To ignore the faulty header just set the relevant "FORCE_*_HEADER" to True in the script header. In your case just set "FORCE_OLD_HEADER = True".
Also note, that extracting contacts in readable format is beyond this script's current scope. This can however be done manually. After you get your data successfully extracted, look for export/data/contacts_DB file. This is a TAR file in which you need to find file "data/data/user/0/com.android.providers.contacts/databases/contacts2.db". This is the contact database as your phone stores it. To actually extract the contacts, you will need to find a way to convert the database to some other format (I have't looked into how the contacts are stored, you will need to do your own googling)

Files extracted but...
hello guys,
I also managed to extract my files using the hack mentioned above (force old headers to TRUE) since by using the default script i was also getting a similar error as above.
However now all of my files (pictures in particular) are corrupted and i cannot seem to be able to open them using several editors.
Anyone else had the same issue?
Thanks in advance!

serlal said:
hello guys,
I also managed to extract my files using the hack mentioned above (force old headers to TRUE) since by using the default script i was also getting a similar error as above.
However now all of my files (pictures in particular) are corrupted and i cannot seem to be able to open them using several editors.
Anyone else had the same issue?
Thanks in advance!
Click to expand...
Click to collapse
Yes I have exactly same issue as you have. All exported files cannot be read.

Hello @tking2000 and @serlal, this script was written based on versions that are available to me. There are two possibilities of what is happening:
1) The file was badly corruped, however the old header stayed itact. (This is the likely case, if you are trying this tool because your phone refused to restore the backup)
2) The file was made by version that does something differently and the script doesn't account for it.
In the first case there is nothing anyone can do (short of trying some corrupted file recovery software hoping the corruption is only minor)
In the second case, you can try sending me a PM with following information, so I can have a look and confirm whether this is truly the case (and improve the script)
1) The script's output
2a) The <Version> and <Attr Name="settingVersionName"> from the <Common> part of the header
2b) If you aren't concerned about privacy or it is just easier for you: just send the whole header file
3) The version of the com.lge.bnr app (LG Mobile Switch); this is not necessary, but may help me
Feel free to censor anything you are sending with [CENSORED] (like your email or filenames)

I have no experience with python, but I have one of these lbf files and I'm trying to figure out how to make it work and all I get is couldn't get library pycryptodome. How do I get Python 3.8.3 to get this library?

Multipart backups
Hello everyone,
I just finished an update, that should enable the tool to extract backups that have multiple files.
If you received "Exception: Invalid header length." in the past and your backup has multiple files, try the script now; your backups are likely to extract correctly now.
This is the likely case for: @Ninja_Toad @serlal and @tking2000
This was made possible thanks to @cheesyzombie who sent me the data I asked for.
@kamikazkid, You don't need to know Python to use this tool, the easiest way is to enter the commands in quotes on the github page into the console (To open the console Shift+Right click in the folder with backups and choose "Open command window here").
Just:
Download the zip on the github page (Under "Code" button)
Make sure the extract.py and your backup are in the same folder
Follow the usage, skip the 4th step and replace xxxxxx to match your backup name.

I figured it out, it was something with windows 10 and the version of python installation I used. Microsoft garbaging things up as usual, thanks for the help though. Thanks, I recovered my files from my old phone successfully.

I have no experience withc++, but I have one of these lbf files and I'm trying to figure out how to make it work, it is a great this forum

Is there a way to import the _DB files (contacts, messages, call logs etc.) back to a phone?
I asked before:
https://forum.xda-developers.com/lg-g3/general/restore-backup-lbf-lineage-17-t4172483#post83644281

Thank you so much for this! I was able to extract the Messaging data from an LG G8 ThinQ. Does anybody know the easiest way to view that data on my PC?

Hi, you can follow these steps to extract data fron _db files:
1) Send the _db files like contacts_db, messages_db to any android phone
2) Double tap on the file and open it using File Manager (which is by default on Mi phones)
3) The file will now open as a folder. Continue opening the folders until you reach to the last file.
4) The last file is the database file. It won't open directly. Select the file and extract the file.
5) Now transfer the file to computer and you can now view it using SQLite DB Browser if its a contacts file or you can use any VMG reader if its a messages file.
Hope this helps you.

Hi @Mysak0CZ, just follow your instructions on GitHub but when I run the command on Command Prompt, nothing happens. Just like this:
F:\>python -m pip install pycryptodome
F:\>python extract.py LGBackup-201031.lbf
When I enter nothing happened. Kindly help to advise for this.
Thank you

Problem in powershell
Mysak0CZ said:
Hi! Your backup seem to be at least partially corrupted. (The new header position seems to start at 7021235167904623918, however the file is only 597864605 bytes long). Fortunatly there are two headers in the backup and the old one seems to be ok (You got "[+] Header read OK" for the old header). So you can try ignoring the "new" header.
To ignore the faulty header just set the relevant "FORCE_*_HEADER" to True in the script header. In your case just set "FORCE_OLD_HEADER = True".
Also note, that extracting contacts in readable format is beyond this script's current scope. This can however be done manually. After you get your data successfully extracted, look for export/data/contacts_DB file. This is a TAR file in which you need to find file "data/data/user/0/com.android.providers.contacts/databases/contacts2.db". This is the contact database as your phone stores it. To actually extract the contacts, you will need to find a way to convert the database to some other format (I have't looked into how the contacts are stored, you will need to do your own googling)
Click to expand...
Click to collapse
Hi, I am trying to extract via your method but in the power shell in sending the command "python extract.py LGBackup_1x0x07.lbf" it does not execute and brings me back the initial path line. Do you have solutions in this regard? Thanks in advance
Ps. I use Win10

Worked so well. I only needed to recover images and audio recordings after realizing that LG backup tool won't restore on other brand phones.
Thank you!!

Mysak0CZ said:
Hello, recently I looked into LG backup and restore application using static analysis.
Note: this is based on version lg bnr 8.0.26 from 2020-02-13, tested with 7.0.20 from 2008-12-31
Here is the extract tool:
https://github.com/Mysak0CZ/LBFtool
And here is the technical part (you don't need to read this, if you just want to extract your data):
The file contains 2 headers (one of them is referred to as "new")
These are XML documents encrypted using AES/ECB/PKCS5Padding + SHA256 and null IV with static password
Code:
cd562f61-5399-3978-ac76-7c54b0508010
And they look like this:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<BackUp>
<Common>
<Name>LGBackup_200217_1</Name> <!-- Name of the backup file -->
<Id>Randomly generated UUID</Id> <!-- Id of the backup file -->
<IsNew>True</IsNew> <!-- IsNew is same in new and old header -->
<Version>1.00</Version>
<Attr Name="product_model">LM-X525</Attr>
<Attr Name="os_version">9</Attr>
<!-- And other attributes including google email -->
</Common>
<BackupItem Category="settings_DB" StartOffset="59392" DataSize="136626" />
<BackupItem Category="MEDIA_DB" StartOffset="901169" DataSize="78393344" FileName="external.db" />
<!-- Other backuped items -->
</BackUp>
The "new" header has more information than original header, but same structure.
The attribute "IsNew" signals the presence of the new header. (and is True in both files)
All numbers inside file are in Big-endian.
And this is the structure of the file (decimal position):
Code:
0: Unknown - 0x00 for me
1: Int32 - size of the new header
5: Int64 - offset to the new header (usually 13)
13 ... EOF-33: data
EOF-33: 20 bytes - SHA1 hash of the password (if present, otherwise still data)
EOF-13: Int64 - offset to the (old) header
EOF-5: Int32 - size of the (old) header
EOF-1: Bool - Lock byte - the file is (0x01) or isn't (0x00) protected by password
The password is only checked during import, not used to encrypt data (so to remove password, just set the last byte to 0)
Every "BackupItem" entry has a "Category".
Special case is Category="APPLICATION" with this structure:
Code:
<BackupItem Category="APPLICATION">
<APPLICATION Name="com.lge.ime" StartOffset="19221586" DataSize="2385983" Package="com.lge.ime">
<Attr Name="com.lge.ime">com.lge.ime</Attr>
</APPLICATION>
<APPLICATION Name="org.exmple.app" StartOffset="1429652009" DataSize="3420160" FileName="org.exmple.app.tar" Package="org.exmple.app">
<Attr Name="org.exmple.app">org.exmple.app</Attr>
</APPLICATION>
<!-- ... -->
</BackupItem>
All applications are tar files that contain 2 folders:
app - contains the apk file
data - application's internal data
All other items have:
StartOffset - Offset inside file
DataSize - Self explanatory
FileName - Not all have this, but most do; it is the original name of the backuped file.
The files are not compressed nor encrypted.
Footnote: This is my first post on XDA, so sorry for any mistakes I made (including my english) and feel free to correct me/give me advice.
Click to expand...
Click to collapse
ok I am so lost. And here is the technical part (you don't need to read this, if you just want to extract your data):
that statement would make it appear easy. well the link only takes you to the code, there is no tool and nothing with "new" as header. so ive tried deciphering what to do for hours now and i cant figure npthing out. rooted and flashed my lgv20 and did a backup with lg bridge beforehand. trying to restore and phone connects to computer fine but doesnt regster in lg bridge so that is why i need to extract .lbf file. can someone make this easier?

hello..it ended up successful.but there is no jpeg or mp4 file on DCIM folder..how can i restore them?should i do somthing with the script? i don't know pyton

Related

incl. ROM now: WM 6 updated Tornado kitchen with SDHC + 28/25MB total/avail storage

This kitchen is a continuation of the Nitrogenious kitchen released at XDA-devlopers.
Nothing has changed from the tools side, only subtle adaptations and altered content.
Thanks go to :
all the experts at XDA-developers (too many to list)
Nitrogenious for releasing his WM6 kitchen and the superb WM6 package contained therein.
The original kitchen is found here: http://forum.xda-developers.com/showpost.php?p=2150690&postcount=1
Mind that usage description has to be taken from there!
Cotulla for publishing the OMAP850_SDHC.dll that can handle SDHC cards and llnhhy for putting the crucial REG setting in his published package for the Tornado.
More details are discussed here: http://forum.xda-developers.com/showthread.php?t=576164
SGregory for revealing (at least I found it there) that "format BINFS" can actuall take a parameter that sizes the BINFS partition and thus opens the path to gain device storage space if the ROM gets smaller!
More details on saving space are outlined here: http://forum.xda-developers.com/showthread.php?t=491240
- Sir.B and geistteufel for the Squeezer batches for UPX. XDA-Develpoer threads are:
original: http://forum.xda-developers.com/showthread.php?t=481880 and post of geistteufel (the one used in this release): http://forum.xda-developers.com/showpost.php?p=3540501&postcount=92
Disclaimer:
This is for educational purposes only!
There will be no support for the published content!
Enjoy and contribute.
Warning:
Mind that from the "cooking" or "kitchen" point of view this is for sure an old base and possibly more elaborate tools and definitely newer OS versions exist. The produced files should not be able to "brick" your device if you correctly set the Image Name to "OS" in the last step of the cooking process. IPL or SPL should never be written unless you know exaclty what you do - the kitchen only cooks the OS part! Writing incorrect content to the IPL or SPL area will brick your device for sure, so be careful!
Motivation:
This effort was only done to get a running stable ROM with SDHC support included that occupies as little storage space as possible
still having mandatory elements on board
filling the remaining space with useful tools
Space saving strategies (order of benefit):
outsource parts to SD card where possible (.NET CF 3.5)
compress files (.exe and .dll) that are not stored as modules with UPX where function permitts.
remove files that are not needed for any function
remove media data (ringtones, pictures) and leave only the bare minimum
resize media data where possible to further decrease size
not done, but possible:
- remove optional packages (additional color schemes) (25k)
- remove empty packages (15k)
- summarize registry tweak packages to just one additional package (edit the .rgu) (few kB)
About UPX and builtin BINFS compression:
My experience is that the BINFS compression shrinks a set of example files (.NET CF 3.5 files that reside in the \Windows path) to about 49% while UPX'ed they only take 32% of their initial size. So on average you may gain 17% of the initial filesize in your ROM. The larger the file, the better is usually the gain.
Mind that UPX will only compress exe/dll files (even if they may be named differently, see Total Commander's *.tfx) and does not compress exe/dll that depend on .NET. It works luckily on the .NET CF core parts itself.
You should not compress (even if compression works, these DLLs will not load later):
resource DLLs
menu extension DLLs (context menus)
Updated and altered content:
added SDHC support by replacing the OMAP850_SDHC.dll with an SDHC capable version initially created for the StrTrk. So far there are no negative impacts visible.
altered the titles of the options to indicate the space they take - also UPX'ed size
moved the oemstartup.dll and the relevant pictures to the folder where the optional sounds already resided
added German T9 to the English T9 - so both is installed in the system if you select this. Mind that an additional language is not eating much space (~70k)
added the SafeInboxExtension as an own option to add, removed the InboxExtension from the combined "Group SMS + ..." menu
put moBlue to the latest version (2.1) and adapted rgu content.
added an option for registry changes called "tobbie GUI tweaks" that sum up everything that I think is useful (smaller menus, fonts, scrollbars, value for gamma).
updated the TotalCmd to the latest released version
created several additional packages from the previous default content (CeleTask, ClearType Tuner, ClockOnTop, ComManager, Dopod SIM Manager, .NET CF 3.5, OMAPClock, OxiosAlarms, OxiosMemory)
added a new package for .NET CF 3.5 where the GAC_* files have to reside on SD card. The complemtary ZIP containing the files to copy on SD card are located in the "_Changes" directory.
added the GPSID Settings tool to the GPSID directory
included the SP1 fixes supplied by Nitrogenious (FakeCursor not included)
changed several tweaked menus in the settings -> system tweaks
added options for volume setting of the initial beep and voice tag to the Voice Tag menu in system tweaks, changed category of the [HKEY_CURRENT_USER\ControlPanel\Sounds\VRecBeg] from "Notifcation" to "System" so that only the sounds sent form the VoiceTag are audible when the System sound volume is set to 0.
completely UPX'ed the SYS\MMSCAMCLKSTK *.exe and *.dll (except 2)
replaced the htcmidi.avd with the WM5 version to get back good midi playback,
added the HTCSourceflt.dll (from Nitrogenious SP1) to get midi playback in wmplayer
removed two large files from this directory which are nowhere referenced (CameraRC_*.dll).
updated WM5torage to the latest version (1.90), already setting suitable defaults.
fixed default settings for A2DP
Directory and contents
added a batchfile (you may want to edit) where the %SystemDrive% can be set to any value. This allows to install the kitchen on any drive you like and not on C:\ (the normal systemdrive) as it was mandatory. Mind that the scope of this %systemdrive setting is limited to the batch execution only. You can put the whole environment on a large RAMdisk (~380MB required) - this speeds up the cooking dramatically! A large RAM disk is available from "[ QSoft ] Qualitative Software" (1 year trial for the lite version) - see here: http://members.fortunecity.com/ramdisk/RAMDisk/ramdriv002.htm .
added a subdirectory "_Squeezer" where you find the UPX compression set "Squeezer" also published at XDA-developers (readme contained there). I have used this set to batch compress many files before putting them to ROM.
added a directory "_Changes" where you find the compressed and original versions of the files in equally named subdirectories OEM and SYS like in the WORK path. So in case you want to go back to the non-UPX-ed version they are there. Continuing the UPX-batch directory logic (2_Backup, 3_Compressed) there are further ones (4_removed and 5_changed) to document the changes done to the original content.
The rest of the kitchen is identical to the one Nitrogenious had released.
Download here: http://www.mediafire.com/file/xdiz2xzmote/Tornado_Kitchen_v09_by_tobbbie.exe
Quick Start:
- Unpack to C:\
- read cooking guide at Nitrogen's thread (see above)
- using defaults you get a ROM with: http://www.mediafire.com/file/z3ynij5ynzd/default-settings.gif
- available storage 27,97MB, free after 1.st boot 24,8MB
Correction: If you want to use the moBlue package, please edit the RGU file and add a blank line at the end. Using notepad will ensure that the file stays in unicode format.
Correction-2 (14.10.2010): You will experience that while WMP is playing the backlight will not go off as normal. This can be fixed by replacing the HTCWMPPlug.dll in the \windows directory on the device or in C:\Torn\_Changes\SYS\MMSCAMCLOCKSTK in the kitchen with the attached file.
Please make sure that your device is "SuperCID" before entering the "Format BINFS command. See post 3.
added 20100314: (edited 20100504)
Despite it is really extremely easy to cook your own ROM with the kitchen, let me give you a head start with your old Tornado. I have cooked the default settings to a ROM and added all tools that you need to step from a stock Tornado to the cooked one in a single archive.
Download it from here: http://www.mediafire.com/file/njm040ttoxm/_tobbbie-tornado-WM6(SDHC-NetCF_on_SD).exe
Unpacking it you will find a directory structure:
Code:
_tobbbie-tornado-WM6(SDHC-NetCF_on_SD)
├───1 prepare security
│ ├───1 HTCUnlock
│ └───2 SDA_ApplicationUnlock
├───2 prepare for custom flash
│ └───Utils
├───3 flash latest Radio and SPL
├───4 format BINFS 1b00000
├───5 flash ROM
└───6 copy NetCF to SD
└───Windows
Follow the actions in the directories one-by-one:
You only need to do steps 1 and 3 if you come from an official ROM but Step 2 (lokiwiz) needs only be done once per device.
If you flash another cooked ROM you can start from step 4.
Attention: In case you did not notice yet - the following procedures will completely erase all content that you stored on the device (email, SMS, MMS, ToDo, Contacts - simply everything) - the device will be as if it comes out of the box. So back up your data before you do this!
Here is what to do in detail, how and why:
Prepare security: This means that the restrictive program execution privileges have to be set less firm to allow step 2 to run later.
Connect your Tornado to the PC and let Active Sync connect. First run HTCUnlock-CVS.exe in the directory 1 HTCUnlock. This will install a program on your device. Run the installed program there and restart the device.
After the device has reconnected to Active Sync, on the PC run the program SDA_ApplicationUnlock.exe in the folder 2 SDA_ApplicationUnlock. It should confirm "succesfully unlocked".
Now the device is ready to receive the "SuperCID" that allows to flash any ROM to it, regardless of Operator or Vendor limitations. To be on the safe side later, please enter on the device *#06# and note down the IMEI that the device reports - you will need it later.
This needs only be done once per device - it is a permanent setting that survives all ROM updates.
Go to the folder 2 prepare for custom flash and
make sure there are no files *.bin left from previous device's activities
then execute Lokiwiz.bat. It will prompt you with 4 options:
Code:
U. Unlock
L. Lock
C. CID Unlock (SuperCID)
Q. Quit
--------------------
Type the letter and press Enter:
Input "C" <enter>.
It will copy a program (itsutils) to the device and it should ask you for permission to execute - grant execution and let the batch file continue. You should find 2 new files beside the Lokiwiz.bat (lock-backup.bin and cid-unlocked.bin). Move them to a safe place immediately and do not repeat the procedure or call another option!
Be careful to label these files unambigously (best is to append the device's IMEI to the name - get it with *#06# before and do not use the IMEI printed on the label of the device - as restoring a wrong *.bin file to a device will kill the GSM radio access (Message: Data Crashes, please contact your... when trying to connect to the network with a SIM card inserted).
Now the device is prepared to receive custom ROMs.
Let's first put the last available Radio ROM and SPL (Secondary Program Loader) to the device. Go to the directory 3 flash latest Radio and SPL and execute ROMUpdateUtility.exe. After successfull update the device will restart in the old OS, nothing has visibly changed - you could still use the device as it is, all your data are still there.
Now the preparations start to erase the old OS and flash the new one.
Deactivate USB connections for the Active Sync
Switch off the device and disconnect from USB
Press Camera Key and keep it pushed down while connecting the USB cable to the PC - wait until the 3-color screen appears and release the camera key.
Start ttermpro.exe in directory 4 format BINFS 1b00000
Select Serial and Port USB
Press <enter> in the terminal window, you should get prompt CMD>
enter info 2 <enter> you should see something like:
Code:
Cmd>info 2
GetDeviceInfo=0x00000002
+ SD Controller init
- SD Controller init
+StorageInit
CMD55 failed
+ SD Controller init
- SD Controller init
+StorageInit
CMD55 failed
HTCSSuperCID ' HTCE
Cmd>
The last line must show HTCSSuperCID ' HTCE.
If you see anything else there (e.g. HTCSVODA0504 㱍dHTCE - which is for an Austrian V1240) the lokiwiz in step 2 above did not work correctly. Still you have not destroyed anything (hopefully) - so to get the old OS start up again, enter ResetDevice <enter> - the device will restart and boot again. Think about what went wrong in the previous steps.
The lokiwiz batch file and the tools behind it are very powerful and can kill the GSM radio access of the device. Be careful with the *.bin files and keep those of different devices clearly apart.
In case you see HTCSSuperCID ' HTCE then you can pass the point of no return (after this the OS and all your data are deleted from the device) and enter at the prompt format BINFS 1b00000 <enter>. (The value 1b00000 depends on the ROM size, so if you use a different ROM, the value may also be different.) After a few seconds the prompt returns and the partition where the OS was stored is cleaned up now. The device will not boot beyond the 3-color screen in this state. You need to flash the new OS in the next step - but before this enter ResetDevice <enter> - the device will restart and return to the 3-color screen.
Terminate the tterm.exe, you will not need it any further.
Re-activate USB connections in Active Sync - you may forget it later.
Enter the directory 5 flash ROM and execute ROMUpdateUtility.exe. The procedure looks the same as in step 3 but takes a little longer. Do not get nervous as the time at 100% extends a few minutes. The device will reboot and bring you to the new OS.
The SD card that shall be used in the device needs to have the NetCF 3.5 files copied to the directory \Windows finally. This is NOT on the device but on the card - you can copy it on the PC while the sd card is in a card-reader or when the device has is mounted, there the path is \Storage Card\Windows
If the device had a SIM-Lock and it rejects your SIM, go to the lokiwiz.bat (again move out all *.bin files) and select "U" for SIM Unlock - again move the bin files in the directory to a safe place (but you should never need them). Mind that the "lock_backup.bin" is just a copy of the current encrypted area in the device. So this file is different after each step you completed before. Worse: if you do not save the FIRST lock-backup.bin you can never go back to this state.
Mind that lokiwiz.bat has worked for me on a Telenor Sim-Locked nordic ROM CID-locked QTEK 8310, so it should work for any other device as well. If you get the dreaded "Data Crashes..." message and your restore of the correct lock-backup.bin did not help either - your last resort is the SIM Unlock service here: http://imei-check.co.uk/c600_unlock.php. It costs you some bucks, but they seem to re-create the encrypted area with the matching IMEI of your device putting it in a SIM-unlocked and CID Unlocked state. Cheaper than buying a new device.
After you have sucessfully flashed your ROM - maybe you try cooking one yourself?
The selected default settings fill the ROM up to the last few hundred bytes. Adding options will surely jump over the next MB border and your ROM uploading preparations will have to format BINFS with a larger size.
If you have not read it elsewhere yet, the standard sequence to uplad a ROM is:
1.) cook ROM (OS part)
2.) determine size and format BINFS accordingly
more see this thread: http://forum.xda-developers.com/showpost.php?p=3439787&postcount=1
3.) upload ROM
If you start from scratch - so your device is still "untouched" by any custom ROM, you must prepare your device to allow the loading of a custom ROM. This happens in several steps to overcome the various security levels that try to prevent this:
Application unlock the current operating system. Look for "SDA Application unlock" this runs on the PC and remotely unlocks (via the Active Sync Connection) the security of the Windows Mobile operating system. This allows tools to run that you need for the next step.
Super-CID your device (and check if it worked!). Look for "lokiwiz" ZIP file in the forum here. Despite orginally created for the "Wizard" model, it also works for the Tornado in all respect, so it does the Super-CID and it does the SIM Unlock. I did it myself on a QTEK 8310 with a Nordic ROM and SIM-locked to Telnor.
To check if it worked, connect the device in Bootloader mode to the terminal program and enter "info 2" (without the quotes). It has to show HTCSSuperCID ' HTCE
Do not care about SIM lock yet, you can do that anytime later if necessary.
Good luck!
Thanks!
Thanks a lot. Nice work. Very useful. Could you please post one with a PRO rom, preferably the 6.5 version? Or at least the guidelines to make one?
I will not cook any further - this is why I released the kitchen. The strategies to save space are outlined in detail, so other cooks can take them and incorporate to their ROMs.
For me WM6 is sufficient - I don't need the "goodies" that came after that.
Thanks!
Oh fine. Thanks anyway. What is the perceived space saved from this method? And is there any performance hit?
I've seen that UPX'ing has a lot of performance boost so I'm wondering whether it can be made only to the packages or is it applicable to the exe's and large dll's from the CABs too. Since there are a few applications, which even when added later, install to the device memory directly. In these cases, UPX'ing might be highly beneficial in reducing the size as well as giving a speed boost. Any info on this?
Well, indeed you may think that UPX-ing will decrease performance as the file must be decompressed before running - but the opposite is the case!
you save space (most if installed, a little if in ROM - due to BINFS compression that is there anyway)
you get faster file-read time: This pays for especially well for large files (opera, office, acrobat, TomTom and alike). This will by far gain more than you loose for decompression (which goes directly to memory).
Looking at usual read-speeds of about 1MB/sec and an assumed 10 times faster decompression speed to memory, my feeling is that for speed reasons it will pay best for LARGE files (card and memory installed). Mind that after the file is read and loaded to memory, still the application needs to initialize itself. The last step is the same, no matter if UPX-ed or not.
If you tweak the bits for memory saving on the device it is no harm for anything smaller as well. Usually I stop UPX-ing below 50kB in size, but to have the ROM fit in the MB-frame I wanted to achieve I also had to UPX some smaller files as well. Just compare the directories of Nitrogenious' kitchen release and mine.
Hi,
Thanks for kitchen !
Sorry, I'm a novice in ROM cooking (I just modified a bit a ROM for my HTC Touch, long time ago, but I got no problem with flashing ROMS on HTC devices )
So I got a few questions/remarks.
1. I tried to build a custom ROM, but I got an error after selecting options.
I checked log file (a:\Torn\WORK\temp\log.txt) and I found following message :
Failed to parse value name HKEY_LOCAL_MACHINE\Software\hejhej.org\moBlue!!!
InitRegistry FAILED in file ".\Registry\37771312-772c-4ff9-a0a1-b555ad54a025.rgu" within a few lines of line 10.
ImportFromPackageListStrict: (RGUComp) !ERROR failed importing ".\Registry\37771312-772c-4ff9-a0a1-b555ad54a025.rgu"
wmain: (RGUComp) !ERROR failed building DEFAULT hives
If I uncheck "MoBlue", all is OK, so I think MoBlue package is corrupt.
2. When building with default option, what values to put in nb2nbf (CID and start address) ?
I used same as http://forum.xda-developers.com/showpost.php?p=2150690&postcount=1 (82040000 for start adress and ORG_0401 for CID)
Is this OK?
(my phone is an Orange SPV600, CID unlocked, so I think I can put any value for CID)
3. I didn't really understand how to change ROM size. I checked your thread, but I'm still in the dark.
In nb2nbf, in size column, I got "33357824" = "0x13E20248".
So I used "format BINFS 014000000". Is the the way to go ?
Thanks for answers.
Answers!
1. Even I'm unsure about that.
2. Yes. Just select the OS option and it will fill the address by itself.
3. Yes. That is the method I follow. AFAIK, convert the bytes into it's hex equivalent and choose the nearest <higher> hex number with 5 0's at last.
And from what I understand from his post, if you have a ROM with 29.1 mb size, either reduce it to (29 mb - 64k) or add some apps and increase it to (30mb - 64k), to make the optimum use of the available space.
Hi AlainL,
...will have to look after the moblue part - strange, possibly the wrong format of the file (not unicode stored). The content should be ok. I fixed it after updating from the old moBlue inside the old kitchen to the 2.1 version copying the Registry content of the moBlue branch.
Regarding the address it is easy: when you select "OS" and click in the address field, the address is selected automatically - this is the right one.
Your assumption on the format BINFS <size> is correct. This is the way to format it. But your calculation is wrong. The Hex size of your value is 1FD0000 and thus you have to format with 2000000 or your device will not boot after flashing.
The solution to the moBlue problem in the .rgu is very easy.
Edit the .rgu file and add a blank line at the end - that's all.
Editing .rgu files
Just to be on the safer side:
Don't forget to save it in the Unicode format too. Turn off word wrapping while editing .rgu files.
They are all unicode - so if you edit with notepad it should keep this format.
Special attention is required when adding "Multistring" values to the registry. The can be imported as hex - and this hex code must be in unicode format, so 2 bytes per character.
So when exporting the values from the registry to add them to an .rgu package you must take care of this. Took me some tries until I had the .NET CF 3.5 with separated GAC_* package running.
tobbbie said:
Your assumption on the format BINFS <size> is correct. This is the way to format it. But your calculation is wrong. The Hex size of your value is 1FD0000 and thus you have to format with 2000000 or your device will not boot after flashing.
Click to expand...
Click to collapse
Hi
Thanks all for your answers.
I used the right number.
I just made an error while writing in this thread ("0x13E20248" = "333578824").
This cooking works very well.
I just got a problem, maybe someone can help me.
I added Esmertec Java and installed opera mini 4.2, opera mini 5 beta 2 (latest) and opera mobile 10 beta 2 (latest too).
All is OK with 4.2, but with 5b2 and 10b2, I can't pass license agreement screen, because I can't click on "Accept" button (nor "exit" one), neither with left or right menu button or with cursor.
I thinks it's Opera's fault, but if anyone got a suggestion ...
Finally, I got a question :
Now, I'm using a SPV C600 (Orange), and it doesn't have WiFi.
I plan to buy either a XPA1240 or Qtek 8310 (used, quite cheap), but I need to know if WPA is supported.
I can't check by myself, cause every time I try something related to WiFi, I end with an error "Driver not loaded".
I do not cook java in the device but have it installed on SD card - along with the midlets it will take later. The package I use is called "JBEDROSE" (20080813.2.1) and comes from the Vox forum. I have no problems using later versions of Opera there (including 5ß2). I suspect the accept requires a network connection to the opera server (WIFI or AS), so maybe this is your problem?
The WLAN support WEP, WPA, WPA2 and should also cover hidden SSIDs (but I don't use it). The Reg-Tweak "optimal performance for WLAN" is actually putting the WLAN in a mode that will NOT do continuous transmission (and drain your battery real fast - like in WM5) but in a mode that saves battery without affecting performance.
The prices for used 1240 or 8310 are rising at ebay currently
tobbbie said:
I do not cook java in the device but have it installed on SD card - along with the midlets it will take later. The package I use is called "JBEDROSE" (20080813.2.1) and comes from the Vox forum. I have no problems using later versions of Opera there (including 5ß2). I suspect the accept requires a network connection to the opera server (WIFI or AS), so maybe this is your problem?
Click to expand...
Click to collapse
I tied both with AS or via EDGE/GPRS connected (when loading EULA, netwok is required).
It looks like Opera 5b2 and 10b2 didn't recognized both menu button.
I will still do some tests.
tobbbie said:
The WLAN support WEP, WPA, WPA2 and should also cover hidden SSIDs (but I don't use it). The Reg-Tweak "optimal performance for WLAN" is actually putting the WLAN in a mode that will NOT do continuous transmission (and drain your battery real fast - like in WM5) but in a mode that saves battery without affecting performance.
Click to expand...
Click to collapse
Thanks, great
tobbbie said:
The prices for used 1240 or 8310 are rising at ebay currently
Click to expand...
Click to collapse
Because of your excellenet kitchen ?
Opera 5ß2 works on my Jbed (non cooked as I wrote). The opera 10ß2 is native WM - but said to not support smartphones (non-touchscreen) well, so I stick with Opera-mini for the occasional browsing I do there.
Now the default settings are contained in a ready cooked ROM - enjoy!
Appended to the first post:
added 20100314: (edited 20100504)
Despite it is really extremely easy to cook your own ROM with the kitchen, let me give you a head start with your old Tornado. I have cooked the default settings to a ROM and added all tools that you need to step from a stock Tornado to the cooked one in a single archive.
Download it from here: http://www.mediafire.com/file/njm040ttoxm/_tobbbie-tornado-WM6(SDHC-NetCF_on_SD).exe
Unpacking it you will find a directory structure:
Code:
_tobbbie-tornado-WM6(SDHC-NetCF_on_SD)
├───1 prepare security
│ ├───1 HTCUnlock
│ └───2 SDA_ApplicationUnlock
├───2 prepare for custom flash
│ └───Utils
├───3 flash latest Radio and SPL
├───4 format BINFS 1b00000
├───5 flash ROM
└───6 copy NetCF to SD
└───Windows
Follow the actions in the directories one-by-one:
You only need to do steps 1 and 3 if you come from an official ROM but Step 2 (lokiwiz) needs only be done once per device.
If you flash another cooked ROM you can start from step 4.
Attention: In case you did not notice yet - the following procedures will completely erase all content that you stored on the device (email, SMS, MMS, ToDo, Contacts - simply everything) - the device will be as if it comes out of the box. So back up your data before you do this!
Here is what to do in detail, how and why:
Prepare security: This means that the restrictive program execution privileges have to be set less firm to allow step 2 to run later.
Connect your Tornado to the PC and let Active Sync connect. First run HTCUnlock-CVS.exe in the directory 1 HTCUnlock. This will install a program on your device. Run the installed program there and restart the device.
After the device has reconnected to Active Sync, on the PC run the program SDA_ApplicationUnlock.exe in the folder 2 SDA_ApplicationUnlock. It should confirm "succesfully unlocked".
Now the device is ready to receive the "SuperCID" that allows to flash any ROM to it, regardless of Operator or Vendor limitations. To be on the safe side later, please enter on the device *#06# and note down the IMEI that the device reports - you will need it later.
This needs only be done once per device - it is a permanent setting that survives all ROM updates.
Go to the folder 2 prepare for custom flash and
make sure there are no files *.bin left from previous device's activities
then execute Lokiwiz.bat. It will prompt you with 4 options:
Code:
U. Unlock
L. Lock
C. CID Unlock (SuperCID)
Q. Quit
--------------------
Type the letter and press Enter:
Input "C" <enter>.
It will copy a program (itsutils) to the device and it should ask you for permission to execute - grant execution and let the batch file continue. You should find 2 new files beside the Lokiwiz.bat (lock-backup.bin and cid-unlocked.bin). Move them to a safe place immediately and do not repeat the procedure or call another option!
Be careful to label these files unambigously (best is to append the device's IMEI to the name - get it with *#06# before and do not use the IMEI printed on the label of the device - as restoring a wrong *.bin file to a device will kill the GSM radio access (Message: Data Crashes, please contact your... when trying to connect to the network with a SIM card inserted).
Now the device is prepared to receive custom ROMs.
Let's first put the last available Radio ROM and SPL (Secondary Program Loader) to the device. Go to the directory 3 flash latest Radio and SPL and execute ROMUpdateUtility.exe. After successfull update the device will restart in the old OS, nothing has visibly changed - you could still use the device as it is, all your data are still there.
Now the preparations start to erase the old OS and flash the new one.
Deactivate USB connections for the Active Sync
Switch off the device and disconnect from USB
Press Camera Key and keep it pushed down while connecting the USB cable to the PC - wait until the 3-color screen appears and release the camera key.
Start ttermpro.exe in directory 4 format BINFS 1b00000
Select Serial and Port USB
Press <enter> in the terminal window, you should get prompt CMD>
enter info 2 <enter> you should see something like:
Code:
Cmd>info 2
GetDeviceInfo=0x00000002
+ SD Controller init
- SD Controller init
+StorageInit
CMD55 failed
+ SD Controller init
- SD Controller init
+StorageInit
CMD55 failed
HTCSSuperCID ' HTCE
Cmd>
The last line must show HTCSSuperCID ' HTCE.
If you see anything else there (e.g. HTCSVODA0504 㱍dHTCE - which is for an Austrian V1240) the lokiwiz in step 2 above did not work correctly. Still you have not destroyed anything (hopefully) - so to get the old OS start up again, enter ResetDevice <enter> - the device will restart and boot again. Think about what went wrong in the previous steps.
The lokiwiz batch file and the tools behind it are very powerful and can kill the GSM radio access of the device. Be careful with the *.bin files and keep those of different devices clearly apart.
In case you see HTCSSuperCID ' HTCE then you can pass the point of no return (after this the OS and all your data are deleted from the device) and enter at the prompt format BINFS 1b00000 <enter>. (The value 1b00000 depends on the ROM size, so if you use a different ROM, the value may also be different.) After a few seconds the prompt returns and the partition where the OS was stored is cleaned up now. The device will not boot beyond the 3-color screen in this state. You need to flash the new OS in the next step - but before this enter ResetDevice <enter> - the device will restart and return to the 3-color screen.
Terminate the tterm.exe, you will not need it any further.
Re-activate USB connections in Active Sync - you may forget it later.
Enter the directory 5 flash ROM and execute ROMUpdateUtility.exe. The procedure looks the same as in step 3 but takes a little longer. Do not get nervous as the time at 100% extends a few minutes. The device will reboot and bring you to the new OS.
The SD card that shall be used in the device needs to have the NetCF 3.5 files copied to the directory \Windows finally. This is NOT on the device but on the card - you can copy it on the PC while the sd card is in a card-reader or when the device has is mounted, there the path is \Storage Card\Windows
If the device had a SIM-Lock and it rejects your SIM, go to the lokiwiz.bat (again move out all *.bin files) and select "U" for SIM Unlock - again move the bin files in the directory to a safe place (but you should never need them). Mind that the "lock_backup.bin" is just a copy of the current encrypted area in the device. So this file is different after each step you completed before. Worse: if you do not save the FIRST lock-backup.bin you can never go back to this state.
Mind that lokiwiz.bat has worked for me on a Telenor Sim-Locked nordic ROM CID-locked QTEK 8310, so it should work for any other device as well. If you get the dreaded "Data Crashes..." message and your restore of the correct lock-backup.bin did not help either - your last resort is the SIM Unlock service here: http://imei-check.co.uk/c600_unlock.php. It costs you some bucks, but they seem to re-create the encrypted area with the matching IMEI of your device putting it in a SIM-unlocked and CID Unlocked state. Cheaper than buying a new device.
After you have sucessfully flashed your ROM - maybe you try cooking one yourself?
Some updates to the guideline for flashing in the previous and the first post.
be careful with lokiwiz (several hints added)
last resort if you fail to superCID the device: http://imei-check.co.uk/c600_unlock.php
Enjoy - there is no real successor of the Tornado
Thank you for this. I updated my phone because the previous rom was slow and flawed as I discovered. Phone is working great now.

[App] [2.3+] OTA Verifier

Whenever there's an over-the-air (OTA) update sent-out or made available (either leaked or official), there's always a large number of users that end-up reporting that their installations have failed with the infamous and dreaded "E:Error in /sdcard/xxx.zip (Status 7) installation aborted." error message. So, I decided to try to write an app that would help folks figure-out what the issues might be so that they can get their OTA update installed.
The OTA Verifier app can be used to evaluate an over-the-air (OTA) update.zip or other flashable .zip file before you attempt the install or afterwards to help you figure-out why the installation may have failed. OTA Verifier will point-out what files and conditions are being tested so that you can more easily attempt to correct these issues.
Play Market Link (free!): OTA Verifier (beta)
Do I have to be rooted to use this?
No, but rooted devices will have more conditions that the app will be able to test/evalute because it will have access to protected files that non-rooted devices don't.
Will the app actually do the installation or change anything on my device?
No. The app will only evaluate the expressions and conditions contained inside the updater-script file inside the .zip file that's trying to be installed. None of the functions or commands that try to modify your device (i.e., delete/patch/format/extract, etc.) will be evaluated or executed. The app basically operates in "read-only" mode with respect to the .zip file being evaluated.
Will this app work on older devices or flashable .zip files that use the amend update-script files?
No, this app evaluates and interprets the newer edify updater-script files (notice the "r" in "updater").
How do I use this app?
Install and launch the app
Click the "Select File" button and navigate to desired .zip file
Long-press (press and hold) the file entry for the desired .zip file
Click the "Verify OTA .zip" file pop-up
Wait for the app to process the file (should take under a minute, depending on your device)
View the displayed results (text will also be copied to the clipboard)
Miscellaneous info:
1. Savvy root users probably already know that after a failed .zip file installation, you should be able to view the /cache/recovery/log file to view the information about what might have failed. OTA Verifier will try to uncover all of the issues and not just the first one that causes the installation to fail.
2. While I wrote this app principally to evaluate OTA update.zip files, the edify updater-script files are used and written by ROM devs and others who created flashable .zip files. This app can be used to evaluate those file's updater-script files, too.
How does this all work?
Here's the basic outline/structure of what the app does:
1. the .zip file is selected by the user via the file selector
2. list of frozen system apps are identified for later reporting
3. the updater-script file is extracted from the META-INF/com/google/android folder in the .zip file
4. this updater-script file is parsed and converted into reverse polish notation (RPN) for execution
5. edify functions that might modify your device are NOT evaluated; these include apply_patch, delete, delete_recursive, format, mount, package_extract_dir/file, run_program, set_perm[_recursive], symlink, unmount, write_raw_image, etc.
6. note: the update-binary executable is not used by or involved in this app; the edify script interpreter that the app uses was written by me, in Java, specifically for this app
7. the remaining edify script functions that test conditions (such as apply_patch_check, apply_patch_space, assert, concat, file_exists, file_getprop, getprop, greater_than_int, if-then-else-endif, ifelse, is_substring, less_than_int, read_file, ui_print, and various operators (!, !=, &&, (, ), ;, ||, +, ==)) are evaluated using an operand/operator stack from the RPN expression parsed from the updater-script statements
8. expressions that fail (return a null-string) or are bypassed (usually because a resource (file/partition) is protected/secured against read-access) are reported for the user
9. after the entire script has been processed, the results are displayed in a pop-up window on the device and the text of those results are copied to the clipboard
Planned / future features:
- preferences / settings
- logging of the output to a file on the /sdcard
- test if .zip file is signed or not
- display more stats
- "explanation mode" to interpret, in English, what the edify commands are testing
Successfully tested on:
Samsung Galaxy Nexus (CDMA) 4.1.1
Samsung Galaxy Nexus (GSM) 4.1.2
Asus Nexus 7 tablet 4.1.2
Motorola Droid X 2.3.4
The following edify commands/functions/operators are supported:
Code:
[COLOR="blue"]apply_patch_check[/COLOR] // apply_patch_check ( <filepath>, <sha1-checksum> [ , <sha1-checksum> ... ] )
// apply_patch_check ( <compoundvalue> )
<compoundvalue> := [ EMMC : <filepath> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
[ MTD : <dev-block> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
[ vfat : <dev-block> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
<dev-block> := filename in /dev/block
[COLOR="Blue"]apply_patch_space[/COLOR] // apply_patch_space ( <bytes> )
[COLOR="blue"]assert[/COLOR] // assert ( <expression> )
[COLOR="blue"]concat[/COLOR] // conact ( <string> , <string> )
[COLOR="blue"]file_exists[/COLOR] // file_exists ( <filepath> )
[COLOR="blue"]file_getprop[/COLOR] // file_getprop ( <filepath>, <propertyname> )
[COLOR="blue"]getprop[/COLOR] // getprop ( <propertyname> )
[COLOR="blue"]greater_than_int[/COLOR] // greater_than_int ( <integer>, <integer> )
[COLOR="blue"]if then else endif[/COLOR] // if <condition> then <expression> [ else <expression> ] endif
[COLOR="blue"]ifelse[/COLOR] // ifelse ( <condition>, <then-expression> [ <else-expression> ] )
[COLOR="blue"]is_substring[/COLOR] // is_substring ( <string> , <string> )
[COLOR="blue"]less_than_int[/COLOR] // less_than_int ( <integer>, <integer> )
[COLOR="blue"]read_file[/COLOR] // read_file ( <filepath> )
[COLOR="blue"]ui_print[/COLOR] // ui_print ( <string> ) parsed & executed, but output not displayed
[COLOR="blue"]![/COLOR] // logical NOT operator
[COLOR="blue"]!=[/COLOR] // not equals operator
[COLOR="blue"]&&[/COLOR] // logical AND operator
[COLOR="blue"]([/COLOR] // open paren: precedence / grouping
[COLOR="blue"])[/COLOR] // close paren: precedence / grouping
[COLOR="blue"];[/COLOR] // sequence/imperative (terminates & separates statements)
[COLOR="blue"]||[/COLOR] // logical OR operator
[COLOR="blue"]+[/COLOR] // string concatenation operator
[COLOR="blue"]==[/COLOR] // equals operator
[COLOR="blue"]#[/COLOR] // comment line
The following edify functions are NOT supported (the majority of them because they are intended to modify your device):
Code:
[COLOR="blue"]apply_patch[/COLOR]
[COLOR="blue"]delete[/COLOR]
[COLOR="blue"]delete_recursive[/COLOR]
[COLOR="blue"]format[/COLOR]
[COLOR="blue"]mount[/COLOR]
[COLOR="blue"]package_extract_dir[/COLOR]
[COLOR="blue"]package_extract_file[/COLOR]
[COLOR="blue"]run_program[/COLOR]
[COLOR="blue"]set_perm[/COLOR]
[COLOR="blue"]set_perm_recursive[/COLOR]
[COLOR="blue"]symlink[/COLOR]
[COLOR="blue"]unmount[/COLOR]
[COLOR="blue"]write_raw_image[/COLOR]
[COLOR="blue"]is_mounted[/COLOR] // innocuous, but still not supported
[COLOR="blue"]stdout[/COLOR] // innocuous, but still not supported
[COLOR="blue"]show_progress[/COLOR] // innocuous, but still not supported
[COLOR="blue"]set_progress[/COLOR] // innocuous, but still not supported
Keywords:
OTA, over-the-air, updater-script, edify, amend
Screenshots:
updated 9-May-2014
updated to version 1.3
Updated app with root-enabled file browser for navigating to protected/secured directories like /cache to select OTA's update zip file.
Full change log to date:
[v1.3 - 04-Nov-2012]:
- for rooted devices, change file browser to allow navigating and selecting .zip files in protected directories (like /cache)
[v1.2 - 29-Oct-2012]:
- minor fix to handle two reports of null pointer exceptions
[v1.1 - 28-Oct-2012]:
- remove unused SD card write permission
[v1.0 - 28-Oct-2012]:
- initial Play Market release
This is pretty cool.
Any chance you could direct me to an explanation for sha1-size (for apply_patch_check) is though? I'm working on some edify scripting stuff and trying to incorporate some asserts for a specific bootloader (important to what I'm doing), and this is honestly the best reference I've found for it yet.
osm0sis said:
This is pretty cool.
Any chance you could direct me to an explanation for sha1-size (for apply_patch_check) is though? I'm working on some edify scripting stuff and trying to incorporate some asserts for a specific bootloader (important to what I'm doing), and this is honestly the best reference I've found for it yet.
Click to expand...
Click to collapse
You bet!
I'm heading home from work here soon and I'll dig-up the details for you .
(yeah, I scoured the internet for detailed information on Edify scripting as well as reviewed the actual Google/Android code--there's not a complete compendium, unfortunately).
I'll ping you back after a bit...
Cheers!
osm0sis said:
Any chance you could direct me to an explanation for sha1-size (for apply_patch_check) is though?
Click to expand...
Click to collapse
Okay, here's what I know about the apply_patch_check command:
There are basically two formats:
Code:
apply_patch_check // apply_patch_check ( <filepath>, <sha1-checksum> [ , <sha1-checksum> ... ] )
// apply_patch_check ( <compoundvalue> )
<compoundvalue> := [ EMMC : <filepath> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
[ MTD : <dev-block> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
[ vfat : <dev-block> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
The first of which is what I think you typically see in updater-script files where you have a filename, followed by one or more SHA1 digests. Here's an example:
assert(apply_patch_check("/system/app/ApplicationsProvider.apk", "41bb5aaaa2791e68b55622fcca13f0e4efa757b2", "fc845332ae7f706824de73f64ae47f93866ad245"));​
The second format is what I call a compound value format, where the file or partition to be checked and the SHA1 digests to be compared, are all concatenated together in a single, colon-separated value. For example:
assert(apply_patch_check("EMMC:/dev/block/platform/omap/omap_hsmmc.0/by-name/boot:4247552:3f4d4f9549d307d152f8503983ee4ff5f46b0a43:4470784:fbd13c778b34fdb7917c1ccee6389aa9b13a92bd")); ​
In the above, I've colored the sizes in red of file/partition on which to compute the SHA1 checksum (colored in black). I figured this out and verified this by using dd to copy the portions of the partition in question and calculating the SHA1 checksums on it.
My app only supports parsing and evaluating two sets of lengths/SHA1s for the compound format at this time (that's all I've encountered so far in the scripts that I've viewed).
Does that help and/or answer your questions?
Cheers!
Thanks! Yeah that helps a lot, but unfortunately it doesn't seem to solve the problem I've been seeing..
Using dd to grab the partition img I can sha1sum that and that SHA1 doesn't allow it to pass the assert, or even a direct sha1_check of the partition.
Code:
run_program("/sbin/busybox", "dd", "if=/dev/block/platform/sdhci-tegra.3/by-name/USP"), "of=/tmp/bootloader.img")
ui_print(sha1_check(read_file("/tmp/bootloader.img"))); = 8c206a1a87599f532ce68675536f0b1546900d7a (also, bootloader.img is 2097152 in size)
ui_print(sha1_check(read_file("/dev/block/platform/sdhci-tegra.3/by-name/USP"))); = da39a3ee5e6b4b0d3255bfef95601890afd80709
assert(apply_patch_check("EMMC:/dev/block/platform/sdhci-tegra.3/by-name/USP:2097152:8c206a1a87599f532ce68675536f0b1546900d7a")); = Fail
They just don't match and I'm completely baffled by it. I can work around it by using dd to pull the img first and using that for my asserts but it's not as clean as I'd like it.
osm0sis said:
Yeah that helps but I guess unfortunately it doesn't entirely solve the problem I've been seeing.
Using dd to grab the partition img I can sha1sum that and that SHA1 doesn't allow it to pass the assert, or even a direct sha1_check of the partition.
They just don't match and I'm completely baffled by it. I can work around it by using dd to pull the img and using that for my asserts but it's not as clean as I'd like it.
Click to expand...
Click to collapse
Is it for the Gnex or the N7 (I have those same devices)?
I'd be happy to test something for you...just PM me or send me an email ([email protected]).
I don't know if you're using my app to test with, but you can also manually run the update-binary directly on/from your phone (i.e., you don't have to run it in recovery, but you obviously need to be careful what your updater-script does ).
I'm still looking for the exact syntax in my notes, but I'll edit this post when I find it...brb.
~ ~ ~
edit: http://wiki.opticaldelusion.org/wiki/Update-binary shows this:
update-binary <api> <pipefd> <zip>
Click to expand...
Click to collapse
I know I did this during my early testing, but I can't remember the exact syntax...I'll try it and re-re-edit .
Man thanks so much, PM sent.
Sorry for cluttering up your thread with this semi-OT stuff.
osm0sis said:
Man thanks so much, PM sent.
Sorry for cluttering up your thread with this semi-OT stuff.
Click to expand...
Click to collapse
LOL, not a problem...it's all Edify-related and updater-script related...I'm betting that's what'll lead most folks here.
Happy to help .
==================
Detailed Analysis
==================
- you have no frozen system apps (good!)
- there were 2 failed expressions; see below for the details about these tests that will cause your OTA install to fail
- there were no bypassed expressions (good!)
-------------------------------
statistics:
frozen system apps: 0
success count: 526
fail count: 2
ignore count: 1587
partition count: 2
protected count: 2
bypassed count: 0
-------------------------------
updater-script analysis details:
FAILED: line #268:
assert(apply_patch_check("/system/app/XT9IME.apk", "8aba56a4406128e78f5729753252c3d93bc21cb4", "965b437bce65018eeb31ff9a381c3687542099e0"));
----------------------
FAILED: line #1038:
assert(apply_patch_check("EMMC:/dev/block/platform/sdhci-tegra.3/by-name/LNX:5013504:c48f8e86c73fb2c2ba1794f5ec98e27c9e206ed5:5060608:319331fae14fec8a88063751475fce26bae328e0"));
So a question, could this failure above be causing my 32 GB nexus 7 to have system update issues? (Not necessarily XT9IME.apk (which I shouldn't have deleted), more the other one) Is there a fix?
Sent from my Nexus 7 using Tapatalk HD
---------- Post added at 10:44 AM ---------- Previous post was at 10:39 AM ----------
==================
Detailed Analysis
==================
- you have no frozen system apps (good!)
- there were 2 failed expressions; see below for the details about these tests that will cause your OTA install to fail
- there were no bypassed expressions (good!)
-------------------------------
statistics:
frozen system apps: 0
success count: 526
fail count: 2
ignore count: 1587
partition count: 2
protected count: 2
bypassed count: 0
-------------------------------
updater-script analysis details:
FAILED: line #268:
assert(apply_patch_check("/system/app/XT9IME.apk", "8aba56a4406128e78f5729753252c3d93bc21cb4", "965b437bce65018eeb31ff9a381c3687542099e0"));
----------------------
FAILED: line #1038:
assert(apply_patch_check("EMMC:/dev/block/platform/sdhci-tegra.3/by-name/LNX:5013504:c48f8e86c73fb2c2ba1794f5ec98e27c9e206ed5:5060608:319331fae14fec8a88063751475fce26bae328e0"));
Sent from my Nexus 7 using Tapatalk HD
Click to expand...
Click to collapse
So a question, could this failure above be causing my 32 GB nexus 7 to have system update issues? (Not necessarily XT9IME.apk (which I shouldn't have deleted), more the other one) Is there a fix?
modwilly said:
So a question, could this failure above be causing my 32 GB nexus 7 to have system update issues? (Not necessarily XT9IME.apk (which I shouldn't have deleted), more the other one) Is there a fix?
Click to expand...
Click to collapse
Looks like you've updated the boot (LNX) partition in addition to deleting (or renaming/moving) the XT9IMG.apk?
So yes, the OTA will not install until you've put both items back to their expected state.
The fix, of course, depends on exactly what you did to change them in the first place.
version 2.1 uploaded to Play Store
Major re-write of app done for version 2.0 and above.
Recent change log:
[version 2.1 - 6-May-2014]:
- fix issue w/identification of non-existent files
[version 2.0 - 5-May-2014]:
- major app update: edify parsing, RPN processing and execution engine completely re-written
- original core app behavior and functionality remain the same, but code cleanup and re-write should mean better and more robust handling of future OTA updater-script expressions
Click to expand...
Click to collapse
Thanks and let me know if you have any questions.
@scary alien could you please tell me how to get apply_patch_space bytes? i am making ota script and i got almost all, still need to generate:
apply_patch_space(bytes) || abort("Not enough free space on /cache to apply patches.");
Click to expand...
Click to collapse
i just dunno how
ZduneX25 said:
@scary alien could you please tell me how to get apply_patch_space bytes? i am making ota script and i got almost all, still need to generate:
i just dunno how
Click to expand...
Click to collapse
It's just the available space (in bytes) in the /cache partition that you want to make sure is available (i.e., free space) for any operations your updater-script file will do concerning /cache.
For example:
apply_patch_space(1000000) || abort("Not enough free space on /cache to apply patches.");
Click to expand...
Click to collapse
Would verify that there is at least 1MB (1,000,000) free bytes available in /cache.
Does that answer what you're asking?
Lemme know--happy to help if I can.
Cheers!
@scary alien not really, i mean i know how it works, just dunno how to generate proper size in updater, for example:
i create regular ota: multiple .p files and some images, zipped, signed OTA.zip size is 20mb, /patch size is 7 mb, images 10 mb and /system (new files) 3mb, how do i know how many bytes i should set to make this ota install in recovery?
should i summarize .p files size or target (patched apk, jar) files size (would be around 90mb) or what else?
I think the size would depend on when your patch files are cleaned-up...i.e., after each patching operation or at the end. If its at the end, you'll of course need to account to all of the space you might use in /cache.
I don't know of a good way to tell you what the high water mark would be other than testing and recording the output of a "df /cache" command at various points in your updater-script file.
I could do that however I don't see it universal or handy, each ota has different size so it would need more/less free space every time.
There is how Google gets this value : https://github.com/MiCode/patchrom_tools/blob/kitkat/releasetools/edify_generator.py#L131 maybe you will understand better.
ZduneX25 said:
I could do that however I don't see it universal or handy, each ota has different size so it would need more/less free space every time.
There is how Google gets this value : https://github.com/MiCode/patchrom_tools/blob/kitkat/releasetools/edify_generator.py#L131 maybe you will understand better.
Click to expand...
Click to collapse
I think I understood what you meant...there's no easy way to make this universal for each OTA except to evaluate each file that might need to use space resources from the /cache partition.
Even the edify-generator git that you referenced does this dynamically:
releasetools/ota_from_target_files:
Code:
for diff in diffs:
tf, sf, d = diff.GetPatch()
if d is None or len(d) > tf.size * OPTIONS.patch_threshold:
# patch is almost as big as the file; don't bother patching
tf.AddToZip(output_zip)
verbatim_targets.append((tf.name, tf.size))
else:
common.ZipWriteStr(output_zip, "patch/" + tf.name + ".p", d)
patch_list.append((tf.name, tf, sf, tf.size, common.sha1(d).hexdigest()))
[COLOR="Red"]largest_source_size[/COLOR] = max([COLOR="red"]largest_source_size[/COLOR], sf.size)
:
:
:
if patch_list or updating_recovery or updating_boot:
script.[COLOR="Blue"][B]CacheFreeSpaceCheck[/B][/COLOR]([COLOR="red"]largest_source_size[/COLOR])
I have seen it, but here comes largest_source_size:
Code:
largest_source_size = 0
https://github.com/MiCode/patchrom_tools/blob/kitkat/releasetools/ota_from_target_files#L532
ZduneX25 said:
I have seen it, but here comes largest_source_size:
Code:
largest_source_size = 0
https://github.com/MiCode/patchrom_tools/blob/kitkat/releasetools/ota_from_target_files#L532
Click to expand...
Click to collapse
Yeah, that's just the initialization of that variable...it gets updated in the for-loop that I included above.
(you know that's not my tool, software, right?)

[TOOL]TWRP/AROMA SPFlash | MediaTek

AROMA/TWRP version of SPFlash Tools.
This tool is for MTK devices with a GPT layout - (MT6732, MT6752 and their variants).
With some testing it may be possible to add some some support for the older devices using an MBR.
Requirements:Copy SPFlash contents to /sdcard/SPFlash
Rename *scatter.txt to scatter.txt
Run .zip from TWRP​MTK-64bit_SoCs-v*.zip:During testing there will only be info displayed on screen, nothing should be modified. (Unless otherwise stated)​SPFlash-v*.zip:This will install all .img/.bin in SPFlash folder if partition is large enough​Downloads:GDrive Folder​Changelogs:SPFlash
v10 - Readded internal sd - fixed aroma exit/about screen - (a few cosmetic issues introduced).
v8 - Aroma menus updated
v7 - Create 'flashable' zip from backed up partitions.
v6 - Zipped backups added
v4 - Backup + Install fixed -- Aroma "Quit Installation" stalls
v3 - Backup function added - untested -- backup works / install from v2 broken
v2 - Menu Select Added
v1 - Initial Release
MTK-64bit_SoCs
v8 - attempts to fix unrelated menu/alert
v7 - calculations done quicker
v6 - script rewrite/reorder
v1 - v5: Initial test builds​
Only follow menu Partition Related -> Scatter Information
Other menus/option are broken/dead ends...
Other things in the MTK folder:simg2img/img2simg are arm source compiled binaries to handle sparse files
file/magic were taken from @osm0sis's AIK (I can't seem to get file to compile)
MTKsplit is used to split boot/recovery images into it's individual parts. Normal images will split into 3; 1 = img ANDROID! header, 3 = kernel (zImage) and 5 = ramdisk (ramdisk). MTK images split into 5; 1,3,5 as before with 2 and 4 being additional headers.​
XDA:DevDB Information
TWRP/AROMA SPFlasher, Tool/Utility for all devices (see above for details)
Contributors
HypoTurtle
Source Code: https://drive.google.com/open?id=0B8WPOq9wZyKxfktHVWgxbW9nYmtJd1ZWc2RIM1NXUU5pSXpramg0bVBYYUdyRDhid3hwM3c
Version Information
Status: Testing
Current Stable Version: V8
Stable Release Date: 2015-09-03
Created 2015-09-03
Last Updated 2015-09-03
Current StatusMTK-64bit_SoCs-v8 .zip will output the partitions that it deems are incorrectly sized in the scatter.txt and will also check the .img/.bins in the SPFlash folder and determine what should go where and will determine if the partition is large enough for the respective .img/.bin
SPFlash-v1.zip will flash the entire contents of /sdcard/SPFlash folder
SPFlash-v3.zip allows selection of files /sdcard/SPFlash folder to install and has a backup option​
Improvements NeededFor SPFlash-v2 will add a basic menu select option for what you want to flash...
In future versions will add backup option and partition resize
Will start to look at resize...​
Derivative Work - using scatt and part from #1
ScatterfixGenerating a fixed scatter from a 'broken one'
Code:
[size=1]#!/sbin/sh
ldr=`pwd`
scatterfix(){
scatt \${b} \${c} \${d}
part \${p} \${q} \${r}
scttrbfr=`cat "${scatterloc:-$ldr}/scatter.txt"`
while IFS=" " read -r a b c && read -r d e f <&3; do
if [ $a = $d ]; then
if [[ ! ${b} = "DONT_USE" && ${b} != ${e} ]]; then
scttrafter="${scttrbfr//${b}/${e}}"
scttrbfr=`echo "${scttrafter}"`
fi
if [[ ! ${c} = "DONT_USE" && ${c} != ${f} ]]; then
scttrafter="${scttrbfr//${c}/${f}}"
scttrbfr=`echo "${scttrafter}"`
fi
fi
echo "${scttrbfr}" > ${scatterloc:-$ldr}/scatter-new.txt
done </tmp/scatter 3</tmp/parted-new
}
scatterfix
diff ${scatterloc:-$ldr}/scatter.txt ${scatterloc:-$ldr}/scatter-new.txt[/size]
Problems to fixThe code will change any hex value deemed wrong to a new calculated value - there is a chance that 4 values are the same but only one/two are wrong - to fix will need to add a loop through scatterfix until the scatter-new.txt generated is actually correct.​​Resize Userdata/IntsdMore of a precursor - bugs fixed and maintained by @odigitech
Thread located here​
Some Dev. QuestionsBefore adding the resizing partitions (Firmware Upgrade) option:
1. Is the preloader header custom or generic; and does it change when flashing the proloader?
2. Is there an actual function to the BPLGU/APDB files other than some sort of device/system check?​
For v7 trying to add a backup + make installable zip option:
Have done it with basic dd backups; would prefer to use sparse/simg2img but simgimg can't seem to use zipped .img's; have looked into using dat/sdat2img but that would require getting python to run in recovery (unless I'm mistaken).
Have added the simplistic option as v7...
odigitech said:
@HypoTurtle I have made a modified version of MTKsplit with more human-friendly names, its in my Drive folder, if it's any use.
Click to expand...
Click to collapse
Thanks, I have updated mine to support non-MTK header-ed boot/recovery.imgs -- can't hurt to make it human readable I suppose.
MTKsplit will be used in the next uploaded version (v8) in the form of a simplistic porting tool...
Should be uploaded on Monday/Tuesday
odigitech said:
You made it work for non-MTK also? Nice one, is it on your Drive?
Sent from my thl 2015 using XDA Free mobile app
Click to expand...
Click to collapse
Should be... but I'll update it with the human-readable version now.
It's not vigorously tested; and not as robust as the (un)mkbootimg binaries out there but it seems to do the job.
@HypoTurtle: on my Jiayu s3, the Germans have been working on android 5.1.1 for it and so far it is great. I installed beta 3 and the next day they released beta 4. Rather then getting out the laptop, I thought I'd use your script, so I got the files needed, I got your script and set it up as the instructions. But when I ran the script, it didn't find the system.img file, and didn't give it to me as an option. I verified that the file was there on disk, but I was never given a check box to select it for flashing. Any logs I can send you to try and figure out why?
Sent from my KFTHWI using XDA Premium 4 mobile app
AlexZap said:
@HypoTurtle: on my Jiayu s3, the Germans have been working on android 5.1.1 for it and so far it is great. I installed beta 3 and the next day they released beta 4. Rather then getting out the laptop, I thought I'd use your script, so I got the files needed, I got your script and set it up as the instructions. But when I ran the script, it didn't find the system.img file, and didn't give it to me as an option. I verified that the file was there on disk, but I was never given a check box to select it for flashing. Any logs I can send you to try and figure out why?
Sent from my KFTHWI using XDA Premium 4 mobile app
Click to expand...
Click to collapse
Is it just the system.img that doesn't appear? And are you using internal or external SD?
HypoTurtle said:
Is it just the system.img that doesn't appear? And are you using internal or external SD?
Click to expand...
Click to collapse
Just system.img, and external_sd
Sent from my KFTHWI using XDA Premium 4 mobile app
AlexZap said:
Just system.img, and external_sd
Sent from my KFTHWI using XDA Premium 4 mobile app
Click to expand...
Click to collapse
Can you post the scatter; the menu items are just an existence check.
[ if *.img/*.bin from scatter exists in SPFlash folder then you are given the option to flash it ]
(Just noticed that it will stall if you try and install and there are no valid img/bin files present)
HypoTurtle said:
Can you post the scatter; the menu items are just an existence check.
[ if *.img/*.bin from scatter exists in SPFlash folder then you are given the option to flash it ]
(Just noticed that it will stall if you try and install and there are no valid img/bin files present)
Click to expand...
Click to collapse
Screenshot of the folder, and the scatter. I checked it and it looked fine.
AlexZap said:
Screenshot of the folder, and the scatter. I checked it and it looked fine.
Click to expand...
Click to collapse
Ah yes - this problem is caused by the amount of items shown - I have it set to show 3, 5 or 12. It will only show 12 (more than 5) if there are 12 things to flash. From your screenshot you have 7 items to be flashed (trustzone gets flashed twice), so you will miss two items - the second flash of trustzone.bin and system.img.
Hope that makes sense; I can alter it to work with 6+; but you should be able to adjust it to work as described above. the reason I did it this way was that otherwise it would look a bit messy - I suppose I could add a 7 option.
HypoTurtle said:
Ah yes - this problem is caused by the amount of items shown - I have it set to show 3, 5 or 12. It will only show 12 (more than 5) if there are 12 things to flash. From your screenshot you have 7 items to be flashed (trustzone gets flashed twice), so you will miss two items - the second flash of trustzone.bin and system.img.
Hope that makes sense; I can alter it to work with 6+; but you should be able to adjust it to work as described above. the reason I did it this way was that otherwise it would look a bit messy - I suppose I could add a 7 option.
Click to expand...
Click to collapse
Ahhh... Not a. Problem then. I don't really need to flash anything besides system and boot. I just included the rest for completness. I'll take one out and let you know how it goes.
On a side note, any reason for these (3, 5, and 12)? An not just everything that is there?
Sent from my JY-S3 using XDA Premium 4 mobile app
HypoTurtle said:
Some Dev. QuestionsBefore adding the resizing partitions (Firmware Upgrade) option:
Q1. Is the preloader header custom or generic; and does it change when flashing the proloader?
Q2. Is there an actual function to the BPLGU/APDB files other than some sort of device/system check?​
Click to expand...
Click to collapse
A1 - Generic, MTK preloader has been the same since armv7 to armv8. Yes, dd backup copy needs to be clean up of the header & footer, more info then checkout with AlexZap... :good:
A2 - Not all MTK firmware include it, it contains the IMEI refer to here for more info or a single link that explained everything... :good:
AlexZap said:
Ahhh... Not a. Problem then. I don't really need to flash anything besides system and boot. I just included the rest for completness. I'll take one out and let you know how it goes.
On a side note, any reason for these (3, 5, and 12)? An not just everything that is there?
Sent from my JY-S3 using XDA Premium 4 mobile app
Click to expand...
Click to collapse
Not sure if you missed the edit. It was more for cosmetic reasons - I could have had say 12 files being displayed but if there's only one file to flash you would have a screen with a lot of blank entries. 3, 5 and 12 seemed the most appropriate without going down the route of creating a menu for all possible entries.
3 being typical install (system/boot and perhaps blank cache/userdata to wipe); 5 as 3 but with custom etc.
I figured that anyone with more than 5 flashable items in there would have dumped the entire SPFlash ROM which I calculated as ~12 items.
yuweng said:
A1 - Generic, MTK preloader has been the same since armv7 to armv8. Yes, dd backup copy needs to be clean up of the header & footer, more info then checkout with AlexZap... :good:
A2 - Not all MTK firmware include it, it contains the IMEI refer to here for more info or a single link that explained everything... :good:
Click to expand...
Click to collapse
A2. I don't thing it contains the IMEI - but it is the database that the IMEI is coded against; I'm not aware of any devices (other than the P6000) suggesting that you recode the IMEI on an upgrade from KK to LP so there is probably not an issue here.
i never use it myself, typically MDRT is able to recover it on mine, feedbacks from fellow XDA member is that, that is the only way that they manage to recover IMEI on their MTK...
On 2nd thought, BTW, i'm using intel nowadays :laugh: IMEI & calibration info is at a hidden partition, same as Samsung & Qualcomm devices that reside at /efs partition, how did MTK IMEI survive Factory Reset since it is at /data/nvram, i wonder, never really thought about it...
yuweng said:
i never use it myself, typically MDRT is able to recover it on mine, feedbacks from fellow XDA member is that, that is the only way that they manage to recover IMEI on their MTK...
On 2nd thought, BTW, i'm using intel nowadays :laugh: IMEI & calibration info is at a hidden partition, same as Samsung & Qualcomm devices that reside at /efs partition, how did MTK IMEI survive Factory Reset since it is at /data/nvram, i wonder, never really thought about it...
Click to expand...
Click to collapse
On the 64bits at least it is also on a hidden partition (nvram) - and it just gets copied to /data/nvram or something (ie. its not mounted but it's files are there). The DB files etc are present in /system as well as in the full SPFlash ROM so I guess things like MTKEng and apps like chamelephon use that when setting/'fixing' the IMEI.
What are the file formats used by intel stock ROMS? This tool isn't really MTK dependant - it's GPT partition dependant (with a scatter for validity check- I'll post a bear minimum of what a 'scatter.txt' needs to have).
Anyway v8 added - so that files to flash aren't hidden (max. 12 files).
Hmm, further digging seems MTK IMEI is at /dev/nvram, same thing happening on intel, users just never spell out everything, they use the format/ erase flash/ emmc that wipes out the IMEI partition but never tell...
i think on the X3, its not possible as its NOT an Android image file but proprietary fls file which they got it when they acquire infineon i think...
Manual fastboot/ dd backup/ restore & you'll end up with a brick device, ATM, no custom recoveries that boot on the x3, i haven't figure out how to repack its recovery that has three separate different region that requires three proprietary download file that packs it together into a fls file & only their FlashTool_E2 is able to download it correctly...
Can't really understand infineon/ intel for developing such cheap device but with such high end software tools...

[TOOL] little tool support for flashing our moto v0.9.3

hello there,
this tool is windows based. You need the .net 4.5 framework. There is no installer. Simply extract the executable onto your computer.
@xQrzy shared some insight about the flashfile.xml. So i made this tool for reading an image archive and creating proper flash statements. Its very rudimentary but its working.
So, whats to do? Use an image file (zip) or unzip it first. Then execute the program and select the image or the folder. And thats it. There is one tab with some information and on the second tab there is a list of flash operations this image provides. Under options you'll find one checkbox. Uncheck it will generate the full file paths for your image files. The third tab is for the output executing a real flash would return on the command line.
Choose your operations wisely, because eg. erase user data will make a factory reset.
version info
Code:
current version: 0.9.3
- (untested) added experimental flashing. There will be a warning before the actual flashing.
- little rework of the gui.
features:
Code:
- checking MD5 hashes of all files
- (untested) selectable flash commands and running them (thats why i called it installer)
for interested devs:
- its a sharpdevelop project (built with sharpdevelop 5.1 rc1). This project was hacked whithin 5 hours so its not that filled with comments and stuff. Its not on git because of this. You can of course download my sourcecode and compile your own binary if you don't trust me. Which would be clever and cautious.
I'm really happy to be the spark of this idea. ^^
And hope you will make the tool better and better.
aVe2000 said:
hello there,
this tool is windows based. You need the .net 4.5 framework. There is no installer. Simply extract the executable onto your computer.
@xQrzy shared some insight about the flashfile.xml. So i made this tool for reading an image archive and creating proper flash statements. Its very rudimentary but its working. I made this tool capable of executing the generated flash statements but i deactivated this because of the potential problems beeing caused by using this feature. Maybe i will continue on this in future.
So, whats to do? Use an image file (zip) or unzip first. Then execute the program and select the image or the folder. And thats it. There is one tab with some information and on the second tab there is a list of flash operations this image provides. Under options you'll find one checkbox. Uncheck it will generate the full file paths for your image files. The third tab is for the output a real flash would provide in the command line.
Choose wisely your operations, because eg. erase user data will make a factory reset.
Additional features:
- checking MD5 hashes of all files
current version: 0.9.2
Click to expand...
Click to collapse
so it will generate a txt file with the statements to type, it doesn't flash anything by itself, right?
bilbo75 said:
so it will generate a txt file with the statements to type, it doesn't flash anything by itself, right?
Click to expand...
Click to collapse
with version 0.9.3 it tries to flash.
Version 0.9.3 added.

[IDE] ROMS][Windows] Poison Kitchen IDE [Dev Preview 2][2.3-8.1]

Poison Kitchen IDE
Development preview
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
​
Description
A powerful IDE for android ROM development
Runs native on WINDOWS, MacOS themed.
Its powered by my XanderUI class library for the .net framework
Currently Development preview builds are only available, meaning things may be broken, behave incorrectly or other
Click to expand...
Click to collapse
Features
-Full GUI coded in C# for fast runtime
-XanderUI based controls(My C# class library)
-Based on .net 4.5
-Support for every single android version
-Unpack rom from (.zip, .tar, .md5, .img, aml_upgrade_package)
-Pull all required files from device(root needed to copy kernel block)
-Adb, fastboot and drivers installater
-ROM information
-Explorer
-Deodexing(4.4.4 and lower ATM)
-Unpacking the kernel
-Converting file_contexts.bin to standard text
-Kernel explorer
-Repacking kernel
-Logging
-Auto-generating updater-script
-Use generic symlinks if files detected
-Use generic file contets if the kernel does not contain file_contexts
-/data/app auto transition to new rom(somewhat working)
-Wipe data excluding /data/media(/sdcard path)
-Auto kernel block detection
-Auto mounting /system and /data
-Packing rom
-Signing rom
-Updater-script editor
-Extansion support
-Emulated exension scripts(applied via C#)
-Degapps extension
-Deknox extension
-Custom emoji extension
-Enable sony apps extension
Click to expand...
Click to collapse
Features not yet implemented
-root via magisk(systemless) and superSU(standard)
-init.d
-deodexing 5.0 and up
-adbd insecure
-init.d tweaks
-build.prop tweaks
-change display I.D
-add sysrw/ro binary
-logging needs tweaking
-loading rom information updated in the background
-Unpacking RUU as project(HTC)
-Unpacking TFT as project(SONY)
-Unpacking system.dat(sparse)
-Setting your actual device as the project and apply changes in realtime
-/data/app auto transition to new rom
-Kernel may not unpack
-Convert line endings in explorer context menu
-Inbuilt file editor(with EOL auto-detection)
-Boot animation player/changer
-Auto flash rom with ORS(TWRP)
-Bluestacks as rom
Click to expand...
Click to collapse
Settings
Load settings()
-Set default startup project or startup menu
-Enable/disable logging
-Stop logger from detecting files or folders added, deleted, changed or moved
Installation settings()
-Change romname
-Change installation type(autodetect on unpack, User generated, Tool generated)
-Change file contexts method(Auto, extracted from kernel, assumed)
-Enable/disable safewipe
-Enable/disable data/app auto transition
-Enable/disable autodetect ernel blockpath(add path below)
Pack settings()
-Change compression level
-Add signing method(pack into presigned, sign on zip, none)
-Change java heapsize
-Exclude files and folders from being packed/detected by IDE
Default program settings()
Change default program to open image files
Change default program to open video files
Change default program to open audio files
Change default program to open prop files
Change default program to open archive files
Change default program to open jar/apk files
Change default program to open other type files
Current version and updates()
Cleanup settings()
Click to expand...
Click to collapse
License
GNU GPL V3
Downloads
Downloads page
XDA:DevDB Information
Poison Kitchen IDE, Tool/Utility for the Android General
Contributors
Ricky Divjakovski
Version Information
Status: Testing
Created 2018-04-19
Last Updated 2018-04-19
Creating extensions and documentation
Code:
Description -
As extensions are a great adittion to the IDE, whats the use if you cant make your own for automated building?
Information you need to know -
The tool looks for the file "extension.info".
In the extension.info file you will specify the extension name, description and the poison shell script file(.psh).
Package the folder containing the extension as a zip archive
------------------------------------------------------------------
Entension index
------------------------------------------------------------------
Sample extension -
https://github.com/Ricky310711/Poison-Kitchen-Extension-Example
extension.info -
Lines will only be read if starting with "Name", "Description" or "Run"
-------------------- code exmpla
# this sets the extension name
Name:Fake optimizer
# this sets the extension description
Description:Do not apply this to rom, its a fake extension to show an example
# this is the poison script to run(must be in same directory)
Run:FakeOpt.psh
------------------------------------------------------------------
Scripting language(.psh file)
------------------------------------------------------------------
Information
-Must be linux EOL(\n)
-Anything but recognised 1st args are ignored
-Any errors will be ignore by the shell
-Use full paths asif you are in the root(/)
Extracting content to rom
---------- code example
EXTRACT|myFolder
1st arg states we are extracting(copying) a folder to the rom
2nd arg is the folder to extract(must be in same directory as extansion)
NOTE: This will extract the folder to the root of the rom
Changing a line in a file
---------- code example
CHANGE|START|/system/build.prop|ro.product.device=|THIS IS STARTS TEST
1st arg specifies we are changing a file
2nd arg specifies where on a line to search for the string(arg4)
3rd arg specifies the file to change
4th arg specifies the string to look for
5th arg is what you would like to replace the line with
The second arg can be START, CONTAINS or END
Appending a file
---------- code example
APPEND|TOP|/system/build.prop|# A TEST FOR APPENDING TOP
1st arg specifies we are appending a file
2nd arg specifies if we are appending at the TOP or BOTTOM
3rd are is the file to append
4th arg is the content to append
Remove a line from a file
---------- code example
REMOVE|/system/build.prop|# end fota properties
1st arg states we are removing a line from a file
2nd arg specifies the file
3rd arg is the string to look for in the line
Delete a file or folder
---------- code example
DELETE|/system/preinstall
Pretty straight forward, will delete a file or folder called "preinstall" from /system/
Create a file or folder
---------- code example
CREATE|DIRECTORY|/preinstall
arg 1 states we are creating something
arg 2 specifies if a FILE or FOLDER
arg 3 is the file or folder to create
Rename a file or folder
---------- code example
RENAME|/system/bin/am|amRenamed
1st arg specifies we are renaming a file or folder
2nd arg is the file or folder
3rd arg is the new name
Changelog
Development preview 2
-Added pull rom from device in setup
-Smoother nvigation
-XanderUI 1.6.0 integration
-Message boxes are now themed like the app
-Extensions now show progress upon running
-Deodexing up to Android 4.4.4 implemented
Development preview 1
-initial release
reserved3
supports all versions of android and all devices, no manual input needed.
dev preview 3 possibly tommorow
Yay now I can at least try to build my own g6 ROMs. Nice work.
development on hold until the 30th as im on holiday, i have been dedicating little time to this as internet here is extremely slow
Magisk and SU support added aswell as deodexing 5.x > 6.0
Also working on disabling signature verification, adding sysrw(as binary)
hi, im trying out the program, it never completes the extraction, and the progress icon starts at 53% do i need to do anything when trying to start the program.
i haVE TRIed running as administartor and without running as admin.
thanks
Twisted714 said:
hi, im trying out the program, it never completes the extraction, and the progress icon starts at 53% do i need to do anything when trying to start the program.
i haVE TRIed running as administartor and without running as admin.
thanks
Click to expand...
Click to collapse
A dialog will more then likely poppup requesting permission to run imgextractor.exe, be sure to accept that to complete the process
Ricky Divjakovski said:
A dialog will more then likely poppup requesting permission to run imgextractor.exe, be sure to accept that to complete the process
Click to expand...
Click to collapse
i am attaching some pics to what happens.
please advise. thanks
there are a couple more that are insignificant
Twisted714 said:
i am attaching some pics to what happens.
please advise. thanks
there are a couple more that are insignificant
Click to expand...
Click to collapse
Could you please PM me with pics of the firmware your selecting or even a link?
I've gone ahead and tried this. I pointed the app to my system.img (not the fastboot zip) and ended up with "error extracting". Tool looks promising. ROM is available for download here:
http://en.miui.com/download-333.html
I extracted it and used the images/system.img
oreo27 said:
I've gone ahead and tried this. I pointed the app to my system.img (not the fastboot zip) and ended up with "error extracting". Tool looks promising. ROM is available for download here:
http://en.miui.com/download-333.html
I extracted it and used the images/system.img
Click to expand...
Click to collapse
Im going to test now, if the firmware has system.transfer.list, and system.dat, it will not be able to be unpacked until i make a native library or extension to perform the operation as i think it would be extremely stupid for the need to have python installed to run.
The only reason i havent rebuilt smali/baksmali etc is 1. Would take me months, 2. we are modifying a system that relies on java to operate.
in future, hefty operations like unpacking etc, will be coded in ASM/C code for faster operation, Im planning on making the whole project open source to allow changes and fixes submitted by other developers, So if anyones interested let me know and ill upload the source code(Written in C# for the .net 4.5 framework)
Ricky Divjakovski said:
Im going to test now, if the firmware has system.transfer.list, and system.dat, it will not be able to be unpacked until i make a native library or extension to perform the operation as i think it would be extremely stupid for the need to have python installed to run.
The only reason i havent rebuilt smali/baksmali etc is 1. Would take me months, 2. we are modifying a system that relies on java to operate.
in future, hefty operations like unpacking etc, will be coded in ASM/C code for faster operation, Im planning on making the whole project open source to allow changes and fixes submitted by other developers, So if anyones interested let me know and ill upload the source code(Written in C# for the .net 4.5 framework)
Click to expand...
Click to collapse
Not sure if this helps but I've run Imgextractor directly on my system.img with this result:
Code:
Mi-A1-Repository>Imgextractor Mi-A1-Repository\Roms\Fastboot\tissot_images_V.9.5.10.0.ODHMIFA_8.0\images\system.img
ImgExtractor version 1.3.6 <Created by And_PDA (Based on sources ext4_unpacker)>
Extractor for images in EXT2\EXT3\EXT4\YAFFS2\CRAMFS filesystem formats
support SPARSE\SIN\MOTO structure formats
Open image file Mi-A1-Repository\Roms\Fastboot\tissot_images_V.9.5.10.0.ODHMIFA_8.0\images\system.img (size 3221225472 bytes) successfull...
Analize format of file. Please wait...
Found SPARSE FORMAT
Found EXT4 FORMAT
free space in image 188895232 bytes
Extract started. Please wait...
Extract 750 folders and 6306 files successfull
Found 386 symlink files
File stats (uid, gid, permission) save to Mi-A1-Repository\Roms\Fastboot\tissot_images_V.9.5.10.0.ODHMIFA_8.0\images\system__statfile.txt
Extract finish success
Press Enter to continue...
oreo27 said:
Not sure if this helps but I've run Imgextractor directly on my system.img with this result:
Code:
Mi-A1-Repository>Imgextractor Mi-A1-Repository\Roms\Fastboot\tissot_images_V.9.5.10.0.ODHMIFA_8.0\images\system.img
ImgExtractor version 1.3.6 <Created by And_PDA (Based on sources ext4_unpacker)>
Extractor for images in EXT2\EXT3\EXT4\YAFFS2\CRAMFS filesystem formats
support SPARSE\SIN\MOTO structure formats
Open image file Mi-A1-Repository\Roms\Fastboot\tissot_images_V.9.5.10.0.ODHMIFA_8.0\images\system.img (size 3221225472 bytes) successfull...
Analize format of file. Please wait...
Found SPARSE FORMAT
Found EXT4 FORMAT
free space in image 188895232 bytes
Extract started. Please wait...
Extract 750 folders and 6306 files successfull
Found 386 symlink files
File stats (uid, gid, permission) save to Mi-A1-Repository\Roms\Fastboot\tissot_images_V.9.5.10.0.ODHMIFA_8.0\images\system__statfile.txt
Extract finish success
Press Enter to continue...
Click to expand...
Click to collapse
Found the error, the img file is of a wierd format, as its named system.img, it actually contains files from the root, so within the system.img the the root of the image is actually the root of the device rather then the root of the system partition, its extremely odd but none the less extremely simple to fix
Issue is already fixed and will be included in dev preview 3, that is expected for public release in a day or so with much more additions
Ricky Divjakovski said:
Found the error, the img file is of a wierd format, as its named system.img, it actually contains files from the root, so within the system.img the the root of the image is actually the root of the device rather then the root of the system partition, its extremely odd but none the less extremely simple to fix
Issue is already fixed and will be included in dev preview 3, that is expected for public release in a day or so with much more additions
Click to expand...
Click to collapse
Yeah. I found it odd that the system.img had a /system directory in it. Awesome! Can't wait
Ricky Divjakovski said:
Could you please PM me with pics of the firmware your selecting or even a link?
Click to expand...
Click to collapse
its on mega.
https://mega.nz/#!dCA2mAYS!-GrKWuuTNODaYEbt3LWiw4LJzxkrz5wI3T94mQ4PU90
it is a android 6 to android 7 full update. its installed, but i am trying to learn cobble together a rom. this img is for a zoomtak upro, i have found today an image for the uplus/vplus. it has much more stuff in it.
thanks
Twisted714 said:
its on mega.
https://mega.nz/#!dCA2mAYS!-GrKWuuTNODaYEbt3LWiw4LJzxkrz5wI3T94mQ4PU90
it is a android 6 to android 7 full update. its installed, but i am trying to learn cobble together a rom. this img is for a zoomtak upro, i have found today an image for the uplus/vplus. it has much more stuff in it.
thanks
Click to expand...
Click to collapse
I will check it when i get home, if its an upgrade package, i cannot ad support for it..

Categories

Resources