Related
Hello folks,
This is a revised version of the same question posted to another forum, where no usable answer was received. I hope this might reach more knowledgeable people here .
On Allwinner A10/A13 platforms (tablets), the boot process differs significantly in Gingerbread (& earlier) vs. ICS (& later)
I think I have a pretty good understanding of the normal vs. recovery boot process in GB-style firmwares:
- boot.axf detects the special "boot to recovery" button combination based on info (key_min, key_max) found in script.bin
- it goes reading /linux/recovery.ini instead of /linux/linux.ini
- recovery.ini gives the name of the kernel to boot (uImage) and the name of the parameters file: paramsr instead of params found in linux.ini
- paramsr contains the name of the NAND partition to mount as root (the one that contains the recovery) and the proper arguments passed as the kernel command line
- done deal.
However in ICS-style firmware, things happen differently:
The uImage kernel has been replaced by a tertiary bootloader: u-boot.bin, whose job is to load the Linux kernel.
The recovery.ini file is gone, only linux.ini remains.
The u-boot binary does seem to have all the necessary parameters embbeded to boot in normal or recovery mode. But since it is boot.axf that detects the key combination, how does it pass the information "boot in recovery, dude" to u-boot? It's not using the "boot-recovery" string written to the "misc" partition because this information is checked and used by boot.axf itself and not by u-boot.bin as far as I can tell. Since boot.axf gets alls the information it needs from linux.ini I doubt that it has the ability to pass command-line arguments to u-boot.bin (I suspect that the limited environment available at this time has any such notion as process arguments anyway).
Actually the same question goes when "boot-recovery" is written to the "misc" partition: boot.axf detects this, but how is this information passed to u-boot?
Summary: in ICS-style firmware, how is the "boot to recovery" information passed from boot.axf to u-boot.bin?
I've Googled for this and found only one post with someone basically asking the same question, without a reply.
If someone can enlighten me I'd be immensely grateful.
@Lannig - I'm facing a similar issue in trying to figure out how the "boot to recovery, dude" message is passed to u-boot from boot.axf. I've created a thread with the information here: http://forum.xda-developers.com/general/help/allwinner-a20-stuck-recovery-reboot-loop-t2979416
I went through your thread here as well: http://www.slatedroid.com/topic/41091-ics-bootloader-recovery-boot-process-question/, where you seem to have had little success in your venture, could you please update on the details how you got the boot from SD card working. Thanks!
I create bootable USB flash drives based on Fedora Linux or UEFI that have special purpose applications on them. I am able to use these flash drives on Windows 8 tablets that have the capability to select USB as the boot device. Thus allowing the tablet to be booted from this USB flash drive.
I have a Samsung Tab 3 10.1, the device model is GT-P5210 with 4.2.2 OS. My thought is I could connect the micro USB to a USB hub that has one of my bootable flash drives plugged in and boot the tablet from the flash drive.
However, I have searched the forums and wiki for posts on this or related to it but I have not see any.
Can someone point to a post that addresses this?
metaylor said:
I create bootable USB flash drives based on Fedora Linux or UEFI that have special purpose applications on them. I am able to use these flash drives on Windows 8 tablets that have the capability to select USB as the boot device. Thus allowing the tablet to be booted from this USB flash drive.
I have a Samsung Tab 3 10.1, the device model is GT-P5210 with 4.2.2 OS. My thought is I could connect the micro USB to a USB hub that has one of my bootable flash drives plugged in and boot the tablet from the flash drive.
However, I have searched the forums and wiki for posts on this or related to it but I have not see any.
Can someone point to a post that addresses this?
Click to expand...
Click to collapse
The generic boot-loader (*which resides on the chip-set) has no knowledge of USB etc, as the lk (Little Kernel) is more concerned about trapping keystrokes in order to chain-load into recovery or to boot directly into Android environment (When holding Vol+Down key in this instance) - in pseudo-code (this is from the context/aspect of lk, and also, the memory addresses pertaining to how to read the partitions are hard-coded into this lk so it will know how to process the logic!)
The lk kernel is the de-facto standard by Qualcomm for MSM chipsets (Snapdragon) and adopted by manufacturers such as Sony, Motorola, LG, Samsung and can be found in the AOSP source under bootable/bootloader.
if (Is Volume Down key pressed?) then
chain-load kernel from /recovery partition into particular address in memory and jump to it and start execution, in bringing up the recovery environment
else
chain-load kernel from /system partition into particular address in memory and jump to it and start execution in bringing up the Android environment.
end if.
As the kernel within lk is pretty limited, considering that the binary image of the kernel is burned into the chip and therefore no way of modifying it. And also should be mentioned that lk contains the fastboot protocol in preparation for flashing /boot, /recovery, /system and /data partitions. There are two sequences to boot, primary boot and secondary boot as it is:
Primary Boot -> lk (depending on outcome of logic)
Go into Secondary Boot -> /boot or /recovery
Side note: Samsung is fond of the PBL/SBL (Which is Primary Boot Loader and Secondary Boot Loader respectively) in their jargon when it comes to modding. Thing about Samsung, is that, in some handsets, PBL and SBL may be encrypted (Samsung Wave GT-S8500 is one such example, where porting Android to it was nearly impossible to do because of the DRM within the boot loaders which was a nightmare to deal with and made modding it extremely difficult, nonetheless, it is sort of working via an exploit in the FOTA code!)
This is why there are no extra facilities such as OTG functionality or anything else such as serial communications, reading from SDCard, graphics etc as it would make the lk kernel bigger than is intended. In other words, it is the smallest possible size of kernel that is designated to do just the above pseudo-code happen.
Also, another way of looking at it is this, and this is dependent on the Android version - the USB OTG functionality is fully brought up within the Android environment, i.e when the familiar home screen appears, then OTG's functionality is enabled. Unfortunately not the case when looking at it from lk's perspective.
If you're curious, here's the Qualcomm entry on the above lk which is a part of the tiny C source that has ARM assembly included and found in JellyBean's AOSP source in bootable/bootloader/legacy/usbloader/main.c
Source
NOTE:
I'm not a developer or Android expert. All information provided here is copied from different internet sources and is to the best of my knowledge. I'll not be responsible for any harm to you or your device resulting from this.
1. PC BOOT PROCESS
Before diving into Android boot process, let's have a look at Linux PC first.
Power Button Pressed
Power On Self Test (POST); identify the devices present and to report any problems
BIOS / UEFI
Necessary hardware initialization (keyboard, disk etc.)
Disk (MBR)
DOS Compatibility Region code (optional)
Bootloader
Active/boot partition (Boot sector)
Kernel
Initrd / initramfs (init)
Services/daemons/processes
BIOS / UEFI is the first software code that is hard-coded on board and runs after we press power button. BIOS runs in real (16 bit) mode of processor, thus it can not address more than 2^20 bytes of RAM i.e. routines can't access more than 1 MiB of RAM, which is a strict limitation and a major inconvenience.
When creating partitions, MBR is saved in LBA0, GPT header in LBA1 and primary GPT in LBA2-33, LBA34 (35th) is the first usable sector. Backup or secondary GPT is saved in last 33 LBAs, last usable sector by OS is ( Total LBAs - 33 ). Partitioning software aligns GPT partitions at larger boundaries, e.g. at LBAs that are multiple of 2,048 to align to 1,048,576 bytes (512 bytes * 2048 = 1 MiB) boundaries. So first sector of first partition is LBA 2048 and so on.
When a system boots, driver of a filesystem is to be loaded in RAM in order to use that filesystem, but driver is itself a file, inside some filesystem. It's like a chicken and egg scenario. So the solution is to always load (as a BIOS/UEFI standard) the first sector on the bootable storage (0/0/1 C/H/S in older schemes and LBA0 in newer), which is (legacy or protective) MBR. This communication between BIOS/UEFI and storage media is through commands which are specific to host controller e.g. ATA commands for devices with SATA/AHCI interface on PC.
Master Boot Record (MBR)
1st 512 bytes (1 sector) at the start of 1st valid disk
Bootstrap code (446 bytes) + Partition Table (64 bytes)
Executable code: Bootloader 1st stage scans partition table and finds 1st sector of active partition (or may point towards intermediate stage)
Partition table provides information about active/bootable partition (and all others as well)
Small size of 64 bytes limits the number of maximum (primary) partitions to 4
Since bootloader unable to understand filesystem (inodes etc.) yet, so MBR is itself executable
Last 2 bytes are boot signatures i.e. to find immediately if disk/drive is bootable or not and hence switch to the next
DOS Compatibility Region
This stage is specific to legacy GRUB, GRUB 2 (default bootloader on most of modern Linux ditros) splits this stage to stage 2 and 3
31.5 KiB / 63 sectors next to MBR, contains filesystem utilities
Still loaded by BIOS routines (or bootloader may use it's own drivers)
Required by certain hardware, or if "/boot" partition (sector containing stage 2) is above 1024 cylinder heads of disk, or if using LBA mode
Volume Boot Record (VBR) / Partition Boot Record (PBR)
Sector no. 63 (64th sector) and above may contain Volume Boot Record or Partition BR, very similar to MBR
Also called Volume Boor Sector, it may be the first boot sector on any partition
NTFS saves VBR as metadata file name $Boot at first clusters, which also contains cluster number of file $MFT. $MFT describes all files on the volume; file names, timestamps, stream names, lists of cluster numbers where data streams reside, indexes, security identifiers (SID's), and file attributes like "read only", "compressed", "encrypted", etc.
If disk isn't partitioned, it's the first boot sector of disk
Boot Partition (if exists)
In MBR scheme, a partition can be marked bootable / active using a flag, usually the first partition of disk
Windows stage 1 bootloader reads and loads only the "Active Partition" from MBR Partition Table
Bootsector or VBR/PBR is read by stage 1 or 1.5 (2 or 3 on GRUB2) bootloader which loads stage 2 (4 on GRUB2) or actual bootloader
MBR / VBR Contains:
Jump instruction (first 3 bytes) i.e. "goto boot code" command
Filesystem header
Executable boot code, usually contains jump instruction for next adjacent sector(s) containing stage 2 bootloader
End of sector (similar to boot signature)
Stage 1 or 1.5 (or 3 on GRUB2) bootloader reads the filesystem table (like MFT / FAT) on partition and loads actual bootloader as a regular file
Bootloader (Actual)
Loaded by previous bootloader from the filesystem of same partition
Loads all necessary filesystem drivers (if any further required)
Configuration is read from database e.g. /boot/grub/ on Linux (GRUB) and <"System Reserved" Partition>/Boot/BCD on Windows (BOOTMGR)
Windows:
BCD is binary file, can be read and modified by commandline tool bcdedit.exe or GUI tool EasyBCD
NTLDR on XP simply used C:\ as active partition reading C:\Boot.ini
Linux:
GRUB makes use of modules to offer extra functionality for complex boot processes
It can show a boot menu to user if needed or configured e.g. for multi-booting or in safe/recovery mode or boot from USB/Network etc.
Locates and loads the kernel of desired OS and ramdisk in RAM
If GRUB is unable to handle the kernel of an OS like Windows, it can be configured for CHAINLOADING i.e. read and execute bootsector of the partition containing Windows bootloader
'os-prober' helps 'grub-install' and 'grub-update' finding Windows boot partition (System Reserved) by reading bootloader configuration in that partition
Kernel
1st MB of kernel from same partition (/boot) loaded in RAM by bootlader in read mode, then switch to protected mode (32-bit) and move 1MB ahead clearing 1st MB
Then swith back to real mode and do same with initrd (if it's separate from kernel)
Kernel contain ramfs drivers to read rootfs from initrd and mount it
Initramfs
Contains minimal filesystem and modules (required drivers which aren't carried by kernel) to access real rootfs (hard driver, NFS etc.)
udev or specific scripts load required modules
<ramdisk>/init is usually a script which loads necessary drivers and mounts real rootfs
finally init switch_root's to real rootfs and executes <real rootfs>/sbin/init; sysV (traditional), upstart (Ubuntu's initiative) or systemD (the latest widely accepted)
init > getty (on virtual terminals) > login (program) > motd > login shell > bashrc / bash_profileRead more about LINUX CONSOLE & VIRTUAL TERMINALS
UEFI
UEFI can understand filesystem contrary to BIOS, hence no limitation of MBR code (446 bytes)
Needs an EFI System Partition (ESP), preferrably of minimum 550MB
ESP partition is formatted as FAT32 but can understand other filesystems such as FAT12 (floppy), FAT16, ISO9660 (CD/DVD), UDF etc.
EFI firmware reads directly <ESP_Partition>/EFI/<vendor>/<boot_programs> as configured in boot manager (which disk, which partition, which program)
Boot programs make use of EFI firmware or EFI shell or GUI Boot Manager to load kernel
If boot program is just the disk, (no partition and no program configured), then fallback program <disk>/<ESP partition>/BOOT/BOOTX64.EFI is executed
Secure boot feature verifies signature of boot program before loading
Multi-booting is easy, just read different entry from ESP partition unlike relying on single bootloader to chain load all available OS's
EFISTUB feature of Linux kernel allows booting kernel directly as a boot_program
UEFI works better with GPT than MBR
Must read:
ANDROID PARTITIONS & FILESYSTEMS
2. ANDROID BOOT SEQUENCE
There might be a single or multiple bootloaders (to give directions how to boot). For a typical android device (most common Qualcomm SoC / ARM processor), boot sequence is as follows:
BootROM (like BIOS on PC). It's integrated with SoC.
Processors, bootloaders
POST
SBL
Parallel loading related stuff from different partitions.
Application BootLoader (aboot)
Primary Boot Mode (if no Kernel detected or if bootloader/download mode key combination applied)
Bootloader/Download Mode
Secondary boot
Kernel (hardware detection and populating /sys, /dev/ and /proc directories as the processes start) and initramfs (creating rootfs and other pseudo filesystems on rootfs)
Init (first process with PID "1". It initiates further loading of processes and daemons)
System / OS (ROM)
Recovery (if recovery mode key combination applied. It's a kernel with UI to perform basic troubleshooting operations)
3. BOOTLOADERS
Bootloader(s) facilitate the the initial starting up of device by taking control from SoC, performing necessary checks, loading required components and then hand over the charge of booting to kernel. RAM is detected at first stage to start loading configuration of other hardware (like keypad, display etc.) in it.
There exist(ed) multiple bootloaders which are executed by different processors, on different devices with different (partition) names like RPM (PBL), DBL (Device Boot Loader; CFG_DATA or sbl1), SBL2, SBL3 (QCSBL) and OSBL (Operating System Boot Loader) etc.
In a nutshell, on modern ARM devices (Qualcomm SoC):
BootROM / iROM and PBL
iROM run by CPU0 on power button press, loaded in iRAM (before RAM is initialized)
It may set up RAM and execute PBL in RAM or leave this for SBL. iROM/PBL is hard-coded on SoC, written during CPU production process and it's closed source.
On devices (such as open boards or some tablets) which support booting from multiple sources like eMMC/sdcard/USB/UART/Network like a PC BIOS, there is an extra stage between iROM and PBL:
IBL (Initial BL)
It's also loaded in iRAM. Depending on CPU pin settings (hidden and soldered or exposed for manual switching) informed by iROM, IBL passes boot mode selection to PBL and optionally checks PBL integrity if itself e-signed by iROM.
SBL or XBL (Preloader)
IBL calls SBL from eMMC/SDCard which supports LCD output. SBL initializes the DDR RAM, loads the trusted firmware (TZ) and the RPM firmware if not loaded by BootROM. SBL calls the final bootloader after self testing the device.
Uboot is open-source secondary bootloader for embedded devices. However sources of SBL can also be obtained from Qualcomm.
ABOOT (APPSBL; predecessor of Little Kernel)
ABOOT loads Partition Table, kernel, splash screen (logo) and modem. It's also responsible for charging mode and fastboot mode. Memory addresses in RAM for boot/recovery partitions are hard-coded in aboot.
Other examples of final (i.e. just before kernel) bootloaders are uboot (traditional Linux bootloader for embedded devices) or manufacturers' developed BL's like hboot (used by HTC) and redboot etc.
Manufacturers put their limitations (say of network carrier i.e. SIM lock and others) at this stage. USB protocol isn't enough and communication with bootloader to hack such restrictions require special devices (called Flashing Box or Service Box in common language), even sometimes a protocol like JTAG i.e. talk directly to microprocessor.
As a norm, all of these stage-1,2,3... bootloaders are simply called BOOTLOADER. While on some devices there is no bootloader partition at all and bootloader(s) resides on SoC.
Coming back to the booting process, after initializing boot process, bootloader (if it's locked) checks the integrity of boot.img (normal boot) or recovery.img (recovery boot), loads them in RAM and transfers control to kernel offering it with "phys_initrd_start" address of compressed (cpio, gzipped) initramfs.
4. KERNEL & INITRAMFS
Once the kernel is loaded and extracted in RAM by bootloader along with parameters, kernel starts executing. Kernel is in fact a self-contained (static) executable binary, made up of many object files (.o) linked together at compile time. Once the architecture and CPU are identified, architecture-dependent code is executed as per parameters passed from bootloader. Then arch-independent stage is executed which includes setting up drivers (display, touch etc.), filesystems like rootfs, tmpfs, proc, ext4 etc. and initializing console as well (if configured). Here the kernel-space ends and user-space begins (what they call it).
Kernel extracts compressed initramfs in rootfs (which itself is ramfs or tmpfs) and executes /init binary which subsequently reads its configuration files /init.rc and other /*.rc files written in Android specific init language. With the help of kernel, init mounts pseudo filesystems /sys and /proc and populates /dev directory containing device node files. Then it mounts /system and all other partitions including /data (also decrypts it if encrypted) and sets (SELinux security) policies, system properties and environment variables (PATH, EXTERNAL_STORAGE etc.). Additionally init also look after any hardware changes (ueventd) and started services changes (watchdog) occurring dynamically.
Finally init starts the runtime located on the system partition. One of the major last processes started by init is Zygote (Java virtual machine) which compiles apps to run for specific architecture (mostly arm / arm64).
DEVICE TREE BLOB
Device Tree Blob (DTB) - created by DT Compiler (DTC) from DT Source (DTS) text - is a mapping of hardware components on a board/SoC and usually a part of kernel source.
PC hardware usually support hardware enumeration through ACPI i.e. kernel may enquire (probe) the buses - PCI (internal devices), USB (external devices), SCSI (storage devices), HDMI/DVI/VGA (display devices) etc. - which device is connected to it.
Buses on embedded devices (including Android devices) mostly don't support enumeration (hardware discovery) because there are usually fixed set of devices and no option for a different OS to be loaded on device. Therefore OS needs to be informed of all connected devices and this is done by providing a standard DTB to kernel. DTB is provided by SoC / motherboard vendor and is usually a part of kernel source. During boot process, DTB is loaded by bootloader at boot time and passed to kernel so that it can discover hardware and create node points accordingly.
We can view device tree on Adroid device by:
Code:
~# ls /sys/firmware/devicetree/base
~# ls /proc/device-tree
DTB may live on a separate dtb/odm partition as specified by AOSP (and was the proposed solution for ARM based embedded Linux devices before Android's birth) but that isn't widely practiced. Usually DTB is appended to kernel zImage/Image.gz or placed at second stage inside boot.img.
VERIFIED / SECURE BOOT
Ensuring a chain of trust from Power ON up to loading of kernel is with the domain of SoC vendor (Qualcomm, Intel etc.) and OEM's. Injecting some malicious or harmful code at any point during booting is made harder to the extent of impossibility.
To ensure a secure booting chain, PBL verifies authenticity of SBL which subsequently verifies integrity of bootloaders (TZ, RPM, DSP, HYP and aboot) so that to avoid loading of unsigned images (boot, recovery, system and others). TZ, after being loaded by SBL also verifies ABOOT using a hardware-based root certificate.
A bootloader with Verified/Secure Boot implementation verifies boot.img or recovery.img (kernel, initramfs and DTB appended to kernel or on second stage of boot.img) by matching their signature with key(s) stored in "OEM keystore" (some partition like CMNLIB, KEYMASTER or with some other name) which itself is signed by OEM. Some vendors allow replacing/appending this keystore with custom one so that custom signed images can be flashed followed by re-locking of bootloader. A simple detail is given here.
At this stage, the chain of trust is handed over to "dm-verity" key stored in boot image initramfs, responsible for "Verified Boot" process of Google/AOSP. Dm-verity (a part of Verified Boot implementing Linux Device Mapper by Google) is a kernel feature i.e. it comes into action after boot image (kernel and ramdisk) is loaded in RAM. It verifies subsequently loading block devices; /system, (/vendor if it exists) and optionally others.
For details see this, this and this.
Google suggests integrating libavb (native code to verify integrity of boot.img) in bootloaders starting from Verified Boot 2.
Unlocking Bootloader
Read here to know about the risks of BL unlocking.
Unsigned kernel or recovery cannot be loaded unless bootloader is unlocked. To make any modification to OS, a critical piece of process is disabling a security system built into the Android's bootloader (aboot) that protects the read-only partitions from accidental (or intentional) modification for privacy, security and DRM. This is what's referred to as "unlocking NAND" or "unlocking bootloader." You have to firstly unlock bootloader to modify partitions "boot" or "recovery" and to gain root access on /system. If bootloader is locked, you only have write access to /cache and /data partitions. Everything else is read-only on device and bootloader will prevent unsigned images from being flashed to the phone. Unlocked bootloader ignores signature verification check which was initiated by BootROM and then transferred to "SBL" and then to "ABOOT" while loading kernel or recovery.
Some newer devices don't allow unlocking of bootloader directly (FRP) without permission from manufacturer to ensure more security i.e. contents of partition "devinfo" are signed by the OEM and can't be modified without their approval. After having permission, an official method is provided to unlock BL using PC. Still some functions related to Proprietary Content might be lost due to bootloader unlocking.
DRM is used to protect content from being copied.
Certain pre-loaded content on your device may also be inaccessible due to the removal of DRM security keys.
Click to expand...
Click to collapse
Android Rooting
Must Read: Root User and Linux Capabilities: Linux vs. Android
Note: Unlocking Bootloader and Rooting breaks "Verified Boot". It can be dangerous.
In order to perform some privileged task on Android, we need to "root" the device first. Since it's impossible to start a process with elevated privelages from within running Android OS, rooting usually involves running a root process (su-daemon) from boot with all capabilities. Superuser requests are made by any non-privelaged programs by executing "su" binary and permissions are managed by an app.
In early days, rooting usually involved booting into a custom recovery which in turn mounted and modified /system files. Usually some daemon's executable binary was replaced with a custom script. In order to address the OTA and other issues caused by improving security features (SELinux, Verfied Boot, SafetyNet etc.), systemless root method was introduced which is used by latest apps like Magisk. It involves modifying /boot image and putting some files on /data as well. So a new init service is injected fulfilling all necessary requirements of new security mechanisms.
In both cases, a locked bootloader won't boot custom recovery or modifed kernel (boot.img). See Verified Boot. Therefore bootloader needs to be unlocked for rooting.
However it is possible to gain root sometimes without unlocked bootloader but not always.
Other methods of rooting a phone from within a running ROM using some sort of One-Click rooting solution (KingRoot, Z4Root, KingoRoot etc.) depend on some vulnerability or exploit in Android OS. Making such security breaches is getting harder and harder with every new release of Android and with improved defense mechanisms, though it varies for different vendors too. The most prominent was with the release of Lollipop and Marshmallow when systemless method had to be introduced beacuse the previous methods failed to work. When phone is rooted using one of such improper root methods, there is a high probability to face "incomplete root" like messages at some point. If such a rooting method works for your device, it's alarming. This exploit is also a way for malware to enter your device. For examples, see Magisk Installation - Exploits, this and this. A very popular exploit dirty cow was patched later.
In addition to that, there are some hacks for certain devices to flash custom recovery without unlocking bootloader using some kind of Firmware Flasher tool (SPFlasher, MiFlasher etc.) in Download Mode because Download Mode provides access to device even before bootloader/fastboot is loaded. Or if you are expert in coding, you can mimic the custom recovery image look like the factory signed firmware and flash it through stock recovery. But this exploit isn't a universal solution either.
So the proper way to rooting which doesn't need any vulnerability, goes through unlocked bootloader. While buying a new phone this must be considered. Keeping you away from root access and unlocked bootloader goes in favor of vendors. By forcing you to use their ROMs (with bundle of useless bloatware apps), they earn a lot from you - money as well as forced loyalty - by collecting data, showing ads and using a lot of other tactics. Go for a brand that provides kernel source and ability to unlock bootloader (on customer's responsibility and with voided warranty obviously).
FIRMWARE UPDATE PROTOCOLS (BOOTLOADER MODE)
Likewise BL, on every device there might be a single or multiple BL modes with different names like bootloader mode, download mode, emergency mode (EDL), ODIN (Samsung), nvFlash tool etc. When we boot in BL mode, device is stuck on boot logo. Some factory flashers work in these modes such as MiFlasher (Xiaomi) and SP Flash Tool (for MTK devices). Bootloader or Download Mode is accessible even if device is soft bricked i.e. if Recovery and/or ROM isn't accessible.
Download Mode
Download Mode (certain button combination while powering on device; usually Vol. Up + Vol. Down or Vol. Down for longer duration + Power) is an official method used by many vendors to flash factory firmware / updates using Flasher (software). Emergency Download Mode (EDL), as it's called on Xiaomi Devices, can also be accessed through fastboot/adb commands or by using some jigs/jumpers. However, to ensure more security, EDL is disabled on some newer devices.
Download Mode is primary to bootloader mode (at PBL or SBL stage) and can be used without unlocking bootloader.
Odin (Samsung), QPST/QFIL work in Download mode (Qualcomm HS-USB QDloader 9008).
When we boot in Download mode, device is stuck on blank screen.
Fastboot Mode
Fastboot - provided by ABOOT - is a software development tool and a standard communication protocol for Android bootloader. It's an alternate of recovery flashing that works in BootLoader mode (aboot) and comes bundled on most of the recent ARM Qualcomm devices. It's a minimal UI through commandline to interact with device in case of failure or to modify / flash partitions. Some OEM's provide fastboot with limited functionality e.g. 'fastboot oem' commands not working and some devices haven't at all. It's up to the discretion of mobile phone vendor.
Fastboot mode is used to perform operations through commands when device is connected to PC through USB. It works even when phone is not switched on in Recovery or ROM or even if android isn't installed on phone. You can read here what operations we can perform through fastboot mode.
Only NAND (eMMC) and USB modules (drivers) are activated at this stage.
INIT PROCESSES & SERVICES: ANDROID vs. LINUX
FILESYSTEM TREE MOUNTED BY INIT: ANDROID vs. LINUX
RESOURCES:
From the bootloader to the kernel
RESERVED
RESERVED
RESERVED
RESERVED
You have to firstly unlock bootloader to modify partitions "boot" or "recovery" and to gain root access on /system. If bootloader is locked, you only have write access to /cache and /data partitions. Everything else is read-only on device and bootloader will prevent unsigned images from being flashed to the phone.
Click to expand...
Click to collapse
I'm under the impression that unlocking the bootloader is not mandatory for rooting the device.
You can root the device with a locked bootloader and gain full access to /system partition.
NikosD said:
I'm under the impression that unlocking the bootloader is not mandatory for rooting the device.
You can root the device with a locked bootloader and gain full access to /system partition.
Click to expand...
Click to collapse
Yeah I think my brief statement is a bit misleading because rooting is out of the scope of this thread. I have added some details to first post.
Thank you very much for all this useful info.
Some more comments.
A locked bootloader won't boot custom recovery or modified kernel (boot.img)
Click to expand...
Click to collapse
It happens to have a budget Chinese tablet with Oreo 8.0 and MediaTek SoC, which I can root using a modified/patched boot.img with Magisk v17.1 inside of course - I mean full root without problems - keeping the bootloader locked before and after rooting.
In addition to that, there are some hacks for certain devices to flash custom recovery without unlocking bootloader using some kind of Firmware Flasher tool (SPFlasher, MiFlasher etc.) in Download Mode because Download Mode provides access to device even before bootloader/fastboot is loaded
Click to expand...
Click to collapse
The tablet mentioned above, belongs to this category too.
Using SPFT (Smart Phone Flash Tool), I can flash custom recovery TWRP for my device without unlocking the bootloader.
So, I have two questions:
1) Is it rare to have such a device or is it common nowadays to be able to root and flash custom recovery TWRP with locked bootloader ?
2) How is technically possible to patch boot.img for rooting and flash TWRP using SPFlashTool (even in download mode before bootloader) without complains afterwards from bootloader, verified boot, dm-verity and all these safety checks that validate digital signature of Vendor ?
I mean you can do whatever you want before bootloader starts, but how can you escape from security traps after the initialization of bootloader verifications ?
Thank you.
NikosD said:
1) Is it rare to have such a device or is it common nowadays to be able to root and flash custom recovery TWRP with locked bootloader ?
Click to expand...
Click to collapse
I'm not sure how common it is but I must say these are exploits. Developers are making use of these vulnerabilities for positive and negative purposes. But these are not a "long-term" solution for rooting.
2) How is technically possible to patch boot.img for rooting and flash TWRP using SPFlashTool (even in download mode before bootloader) without complains afterwards from bootloader, verified boot, dm-verity and all these safety checks that validate digital signature of Vendor ?
I mean you can do whatever you want before bootloader starts, but how can you escape from security traps after the initialization of bootloader verifications ?
Click to expand...
Click to collapse
That's what my point is. Fastboot code verifies signatures/hashes only when flashing the image and doesn't verify or fails to verify integrity if image is already flashed. This is not the desired behavior so it's an exploit and it should be closed. Letting unsigned images be flashed in Download Mode is another exploit which is common with Chinese vendors as far as I have come across some instances. They don't address "loopholes" seriously. Failure to stop security breaches at or after bootloader level is definitely on SoC Vendor or OEM's part. I have added a paragraph in first post with some useful details and links.
This link explains:
The Qualcomm SoC is analyzed in the previous chapter dload / edl mode, the mode in the firmware image download process does not do any verification, can be directly written into the brush.
Click to expand...
Click to collapse
It's badly translated from Chinese but is informative.
Exploiting Qualcomm EDL Programmers is a complete series on this subject summarized here.
mirfatif said:
Only NAND (eMMC) and USB modules (drivers) are activated at this stage.
Click to expand...
Click to collapse
Hey pal, I'd like to know if you could help me with an issue I'm facing. I have a Moto G5 that isn't booting to any ROM (it either bootloops in bootlogo or in boot animation), and also on TWRP and during the boot animations the device is slow as hell (like 0.5 FPS on TWRP and even less on boot animation; on TWRP the device also takes a few seconds to complete even the simplest tasks - like the press of a button or the swipe of a slider - here's a video that shows differences between how stuff works on fastboot and how slow things are on TWRP, it takes like 2 hours to completely flash a custom ROM, i.e.).
I know much of the issue will be device-specific, but my point (and the reason I quoted that specific part of your OP) is that, on fastboot mode, the device is snappy and responsive. When I press a button it completes the corresponding task immediately, frames don't stutter (not that there are any animations to be rendered in fastboot, but when I switch from one option to another using the volume keys, it does so on screen as it should, with no lag), and so on. Stock recovery also seems to be ok with speed, but it's even harder to measure than fastboot because, in almost 10 years meddling with android devices, I have always found stock recoveries (and CWM in the pre-TWRP times) to be somewhat slow. Stock recovery definitely looks snappier than TWRP, though. Tried several ROMs, both custom and stock, and the issues remain on all of them.
I got to this post by researching if fastboot mode was stored on the same NAND chip as recovery, OS and so on (found out that yes, it's all on the same chip). If it wasn't, I could just assume it was a hardware fault on the NAND chip, and that would be the reason that fastboot was running fine but recovery and OS weren't, but since they're all on the same cell, I can only think that some part of the system (I mean as in every single code that runs on the device, not only the OS) that loads on TWRP and on normal boot, but not on fastboot (and possibly not on stock recovery) are faulty, thus being a software issue (either solvable with just a normal USB cable or needing a flash box).
So, my question is: which are the differences in the parts of system loaded by fastboot and by TWRP? Are there any parts that are loaded by TWRP that aren't loaded by the stock recoveries on most devices?
I know it's a rather complicated question and some stuff might be device-specific, but if there is anything you could tell me that are more generic to every Android device, it would help me a lot. Thanks in advance.
Welcome everyone!
This project has started, becouse we need real solution for the problem. The problem of hard bricked Moto devices. It is like a curse.
When my device bricked I have done solid research, I have gathered many informations and files essential to revive my cellphone but 5 years experience of linux, rooting, compiling kernels and roms weren't enough to make it work.
But nevermind. I am even more determinated and I am asking ALL of You guys here to help me. Together we will come to solution.
Here is what I got, happy reading :
DICTIONARY:
PBL - Primary bootloader of the chip - this is like BIOS for phone so it checks chip for damage and problems and then it tries to load SBL but if SBL is corrupted or checksum doesn't match, PBL invokes Qualcomm HS-USB QDLoader 9008 emergency mode. PBL is hard flashed into SoC and can't be corrupted by firmware.
SBL - Second stage bootloader wich is more advanced than PBL. It initializes phone hardware and ABOOT.
ABOOT - Application bootloader (HBOOT). You probably know this one well. Android botloader.
Full mmcblk0 backup - Backup of whole phone flash storage byto to byte.
blankflash - method of repairing msm phones in 9008 state
programmer.mbn - Special type of software programmer that is being sent to chip in Qualcomm 9008 emergency mode. There it comunicates with pc via firehose protocol. Each phone has set of their own programmers, they are unique to phone and other programmers don't work. These programmers are signed so tampering it results in not working one.
firehose protocol - it is used to tell programmer what operations it must do on chip.
singleimage.bin - this package contains instructions for programmer and set of files it need (for example to replace)
gpt_main0.bin - Partition layout
rawprogram0.xml - instructions for programmer
patch0.xml - I don't know yet
STAR.exe - Application for managing and editing contents of singleimage.bin aka blankflash files
QPST - Flash tool from Qualcomm it basic function is to handle blank-flashing in a better way, also it allows for in-depth debugging of the process
Qualcom Premium Tool - Program made by Mppg Myanmar that is capable of making unlocking bootloader, OEM locks, making backup/restore of chip firmware, handling blank-flashing in VERY specific way (creating instructions for programmer), reading eMMC structure from firmware (can generate gpt layout so very useful!!!), modyfing FW and removing Xiaomi account. It also contains ALL programmers
for more:
https://forum.xda-developers.com/android/general/info-android-device-partitions-basic-t3586565
https://alephsecurity.com/
https://github.com/alephsecurity/firehorse
https://github.com/aravindvnair99/Motorola-Moto-E-XT1022-condor-unbrick
INFO:
1. What causes the brick
I bet 100$ that you hard-bricked your Moto Z Play by installing OTA updates after downgrading firmware. This is only known reason for me at the time of writing this. There is most probable reason why it happens, look:
There are two most common chips on which smartphones are built - Qualcomm and Mediatek. While Mediatek chips are "modification friendly" and simple, Qualcomm chips are somewhat more advanced and have many features that can be enabled or disabled during prorammming in factory. One of them is PBL signature checking. During programming of your phone, proper signatures of SBL are written to it. When someone tries to override default SBL with the new one, it checksums are compared with that stored. If they match, new one is flashed, if not, then update does not happen.
Ok, but what it has to do with brick?!
I explain:
1. You decide to downgrade your firmware
2. During flashing, everything goes "well" (Phone boots), but trully update is partial:
FW in chip is (obviously) more recent that the one you downgrade to, and SBL signature is different (updated), so when it is compared to the signature of SBL from FW you want to flash, it don't match. That don't rise error and flashing continues. Only partition that stays untouched is bootloader, but all other partitions get replaced by those in FW zip. SBL is still compatible with the new partition offsets and partition layout overall so phone functions normally.
3 When OTA is executed, it checks the version of currently installed firware. The most reliabe way to do it is to check checksum of SBL which is pretty logical becouse it's checksum is like "fingerprint" of firmware. Normally, if it would detect the old firmware, OTA would be stopped, but newer SBL tricks it and OTA installs anyway.
4 Results are horrible, becouse OTA does not check GPT table and flashes partitions in bad sectors, corrupting FW.
This causes bootloader to go into Qualcomm HS-USB QDLoader 9008 safe mode.
5 Viola! Hard brick!
2. How to fix it?
That is jolly good question! What we have to do is to reflash full chip firmware. Suprisingly I see some solutions, but those need to be developed:
A) SD-BOOT
It turns out that our fancy chip can probably boot from SD-CARD! The procedure works like this:
- When chip starts, one of the very first things it does is loading the memory, so it can actually work. The trick, is that chip loads it from specific disk, marked with exact name (I don't remember which, but I will do research). Speccially repared SD-CARD can appear with that name, so chip boots from it, not from internal memory. (This trick is proved to work on this model)
How to do it?
- Get full dd of working phone - it must be phone with the SAME chip and very likely the same model
- flash it to SD-CARD of 32GB or more, class 10 speed or higher, directly to card, not partition
- put card in phone, turn it on and wait
- you should see HBOOT
- select fastboot and flash new FW via it
- viola!
!!!THIS IS COMPLICATED PROCEDURE, I WILL MAKE DETAILED THREAD SOON, BUT FOLLOW IT ONLY IF YOU KNOW WHAT ARE YOU DOING!!!
B) FIREHOSE/SAHARA ATTACK
This could be achieved by sending payload via Firehose programmer that would allow to break verification of SBL or somehow allow SBL to be flashed. Now, PBL blocks attempts to update SBL. I have thesis that it is becouse PBL do not allows for SBL downgrade, so it's version must be higher, but we try to flash same version of SBL so it doesn't work. That thesis needs confirmation.
C) CRAFT BLANKFLASH
This would be last resort. It will work for sure, but this method needs knowledge and I don't know if it is doable.
STEP 1: Get white-listed blankflash checksums from OTA (we would need to reverse engineer those)
STEP 2: Break hash
STEP 3: Craft blankflash with needed hash
STEP 4: Flash
NEVER USE BLANKFLASH (ATTENTION!)
DO NOT try any blankflash files. They can make situation a lot worse and even physically (!) dmage your phone.
D) JTAG
Medusa Box etc.
E) Qualcomm Premium Tool
This can even work, but it is untested and there is a slight chance that can worsen state of phone (needs confirming).
The tool is very advanced and I need to gather info about usage, so very probable to be a good solution if we will learn how to use it!
E) METHOD 7
Interesting method from this guy: (7th option, I have contacted him if it is compatibile)
https://github.com/aravindvnair99/Motorola-Moto-E-XT1022-condor-unbrick/blob/master/Unbrick%20methods.md
3. DOWNLOAD
(Links will be aded *soon*)
XDA:DevDB Information
Unbrick Developement for Moto Z Play (addison) Full-Brick, Tool/Utility for the Moto Z Play
Contributors
Bobernator, Artim_96, Camarda
Version Information
Status: Testing
Created 2019-05-03
Last Updated 2019-05-03
Hi, same problem. Did you solve it?
Maybe you already have a loader for Qualcomm "Emergency DownLoad" (EDL) mode.
Maybe you're looking for one.
You know what? A single loader is for more than one device. But it gets hairy with signing and manufacturers and stuff.
So, I've got a beta release utility here. It can (in most cases) identify which model Qualcomm processors a "Firehose" loader is designed for.
First, it's currently a Windows release.
Second, it doesn't work with the older .mbn style loader (since they don't include that information).
So, just go to My EDL page and go to the bottom and download qcomview.exe
Code:
C:\>qcomview.exe poke3.bin
APQ8096
APQ8098
MDM9250
MDM9255
MDM9350
MDM9650
MDM9655
MSM8996
MSM8997
MSM8998
QDF2432
SDA630
SDA636
SDA658
SDA660
SDM636
SDM658
SDM660
You can see the SDM 636 (which is the actual processor on a Poke3.
Obviously, you have to select your own loader.
I've scanned through 200 loaders and I recognize all the processors.
If you see a "???" please quote it.
Edit: Maybe you're saying, "That ain't nothing but a "string" script!" Eh, mostly, but it is more clever and it sorts things.
Thanks for the tool. I have a small feature request, since xbl and elf firehorse programmer use similar structure(I guess), it would be useful if you add a way to check if xbl and programmer are compatible(by comparing cert hashes?).
HemanthJabalpuri said:
It would be useful if you add a way to check if xbl and programmer are compatible...
Click to expand...
Click to collapse
It would be.
On your device you already have a ton of ELF images that have compatible signing.
The problem is, the certs are not identical since the lowest level (farthest away from the root authority) has things like dates and annotations and the bit fields are not the same.
I've not yet figured out how to generate from an ELF file the 256 bit "Hash" that EDL gets out of the device.
To those who don't know yet, I've added more things to this utility. It can check the regular hashes in the ELF files. If your device is not SecureBoot this can be handy if you want to patch. The hashes on the program segments in an ELF file are always checked, the signing is only checked if SecureBoot is on. So, if your SecureBoot is off, you can patch a file, run qcomview /h whatever.elf. As of now it won't can correct wrong hashes but you can simply hexedit in the bigendian values and then double-check with the same command.
Code:
C:\>qcomview /h xbl
64 bit ELF, SHA384
0 00000000 000003f8 8a46a864b9bec352 69b1dadfcac64bfa a388f7bea37d855e 50f55170277c043c 87c862e23709fd96 34bb545ac49a3d64 OK
1 00001000 00001cd8
2 0005cd10 00002ab0 3d2e7c505458e1e7 9070b1957a8f2520 3bbcf288674548f1 7db146a86b314499 5890e1432dbac635 2bad53bfd2960908 OK
3 0005f7c0 00000d64 ac556708059a1315 41e774e34310b89f 3c3f13183b43fda9 9e3a34bd0899da4b bb43c1080a43925f fd8d6a2ecd864e29 OK
4 00076d70 00000000
5 0005cd10 00000000
6 00003000 0004cd04 a81ab8ec59e2dfb1 f2f98e3ac0a9a396 1cd9f0dfb5a5daa5 2cda2f52d4df97c8 bc398b24528fd10f cd47ced08596f61c OK
7 0004fd10 00000000
8 0004fd10 0000d000 e7d03abb34361774 e030039e096b3e25 64519024c5c15666 efecbd8006deaaae b87884e2bdab52cb e06a4a7a4873e1c5 OK
9 0005cd10 00000000
10 00060530 00016838 2ca0423b6e745b5f c69544b947556ff1 9d04792c579d2f53 d480d2fa738cac82 1674ddaab8078071 648cc10f384ec25a OK
11 00376d70 00022000 18bdbbdeac3e92c0 6f3e5f06f5aa91ae d0daa757a375bab6 5e90d4e2a52d8e95 2255d80c76637316 b24736223e0a0bd2 OK
12 0005cd10 00000000
13 00398d70 00048ded 794528234b46757a 3017481198fa8fd6 c9578e6565ec301a f0ab28fbe105c460 c7cc855f93576767 29302c26357a00bb OK
14 003e8490 00000000
15 003e1b60 0000692d 1354b9b55447ffb8 54ea17d1d9f1ea88 c84bd1045a6bd106 3b38df93fa049fa9 c1b245dc6106098a 0450a75bf7e5ce3f OK
16 00076d70 00300000 7341f2cde09d6a5f 53bcb90714f779a5 53c3ffeeff1824e5 437464f4bfcc545f 6719370d5d6c656d df96e81382315405 OK
For you Motorola users running into "range restricted" you can dump the ranges by:
Code:
C:\>qcomview /r motog.bin
Addr LUN Start Count
------ --- -------- --------
008220 0 0 32
008238 0 -5 5
008250 1 0 32
008268 1 -5 5
008280 2 0 32
008298 2 -5 5
0082b0 3 0 32
0082c8 3 -5 5
0082e0 4 0 32
0082f8 4 -5 5
008310 5 0 32
008328 5 -5 5
008340 1 0 2048
008358 2 0 2048
008370 3 0 2356
008388 5 0 2356
0083a0 0 2080 512
0083b8 0 0 256
0083d0 0 -33 33
0083e8 0 131072 284992
008400 0 416064 2048
008418 1 1 1
The UFS table is on top, followed my the eMMC table.
HemanthJabalpuri said:
It would be useful if you add a way to check if xbl and programmer are compatible (by comparing cert hashes?).
Click to expand...
Click to collapse
I've just added SHA256 fingerprint of the root CA to qcomview.
Code:
C:\>qcomview /f loader.bin
5adc6039 dcb297d4 0c55df73 1580248d a9e18b31 ccc43b45 36795313 f82fd430
If SecureBoot is enabled xbl/abl/Firehose must all have the same fingerprint.
(This also goes for the other two dozen ELF files in flash.)
For most devices this SHA256 will be the same that your EDL client prints out as "Hash".
There appears to sometimes be (on newer devices?) a discrepancy between root CA fingerprint and EDL "Hash".
Possibly the EDL "Hash" is the encrypted version?
In any case, all the fingerprints should agree.
Renate said:
Maybe you already have a loader for Qualcomm "Emergency DownLoad" (EDL) mode.
Maybe you're looking for one.
You know what? A single loader is for more than one device. But it gets hairy with signing and manufacturers and stuff.
So, I've got a beta release utility here. It can (in most cases) identify which model Qualcomm processors a "Firehose" loader is designed for.
First, it's currently a Windows release.
Second, it doesn't work with the older .mbn style loader (since they don't include that information).
So, just go to My EDL page and go to the bottom and download qcomview.exe
Code:
C:\>qcomview.exe poke3.bin
APQ8096
APQ8098
MDM9250
MDM9255
MDM9350
MDM9650
MDM9655
MSM8996
MSM8997
MSM8998
QDF2432
SDA630
SDA636
SDA658
SDA660
SDM636
SDM658
SDM660
You can see the SDM 636 (which is the actual processor on a Poke3.
Obviously, you have to select your own loader.
I've scanned through 200 loaders and I recognize all the processors.
If you see a "???" please quote it.
Edit: Maybe you're saying, "That ain't nothing but a "string" script!" Eh, mostly, but it is more clever and it sorts things.
Click to expand...
Click to collapse
Hello , Renate
I am using you edl.exe programme. it work fine but i would like to know that the tool has any features to flash using xml file or not ? and it is support ufs provisioning or not ? Please confirm
noob9t2 said:
Please confirm
Click to expand...
Click to collapse
Yes, it does UFS (with the /u flag).
No, it doesn't do these XML files. I find the whole idea a bit overblown.
If you're in the habit of overwriting every partition on your device often, simply:
Take the XML file and delete all the redundant stuff besides 1) partition name, 2) image filename.
Add in edl /w /p on each line.
Execute it as a batch file.
Thank You Renate for reply. we flash ufs chip using qfil after flashing on qfil, we need to flash patch file and check ufs provisioning to boot the device properly. On your tool, anything need to do after writing a partition. if i write a single partition, phone will boot normally ?
noob9t2 said:
If i write a single partition, phone will boot normally?
Click to expand...
Click to collapse
Sure, if you didn't break anything.
The reboot command is edl /z
Ha! You motivated me to track down why some devices need you to do that command twice.
I just fixed it.
Download the special Valentine's Day release of edl.exe (from the usual place).
noob9t2 said:
We flash ufs chip using qfil after flashing on qfil?
Click to expand...
Click to collapse
So, if you're using QFIL there's a loader somewhere that you're using. Find it.
Please can you explain how the patch for the loader works
roulo said:
Please can you explain how the patch for the loader works
Click to expand...
Click to collapse
Loaders are made by phone manufacturers from standard editions of xbl (the secondary loader) released by Qualcomm.
Sometimes they put in restrictions (like Lenovo/Motorola), sometimes they put in authorization (like OnePlus).
Sometimes there are two different versions, one with full capabilities, one without.
The word "patched" gets used often for the full capabilities loader.
Patching a loader yourself is not that difficult, the problem is that loaders must be signed and you can't do that.
Many components on Qualcomm SoC phones are signed.
This ensures a "chain of trust".
The only way that you can patch something is if your device does not have SecureBoot enabled.
If you know of a phone without SecureBoot, tell me and I'll buy a case of them.
I never had time but here is a starting point.
https://forum.xda-developers.com/t/k40-bricked.4538285/post-87978383
alecxs said:
I never had time but here is a starting point.
https://forum.xda-developers.com/t/k40-bricked.4538285/post-87978383
Click to expand...
Click to collapse
What I could read of that was talking about analyzing Firehose loaders for vulnerabilities, which you can.
I've largely disassembled a "restricted" Motorola Firehose loader and could patch it easily.
Still, unless some Motorola employee goes rogue I don't see how I could sign it.