Secure boot still enabled even after being disabled in the UEFI settings

Secure boot still enabled even after being disabled in the UEFI settings - Microsoft Surface

Hi,
I would like to install a live Kali with persistence on the Surface pro4 internal sdc.
So I've installed ReFind on the system EFI as mentioned in this article:
ceres-c.it/booting-kali-surface
Then I have disabled all the security options of the UEFI bios (no certificate, no TPM), and when I boot on from the internal sdc, I got something like "refind secure boot failure!"
From my understanding (and reading) this should not occur when security options of the UEFI Bios have been disabled.
Did some of you experienced this kind of trouble?
Anyway, advise are welcomed
Regards

Related

[Q] Boot Galaxy Tab 3 10.1 from USB

I create bootable USB flash drives based on Fedora Linux or UEFI that have special purpose applications on them. I am able to use these flash drives on Windows 8 tablets that have the capability to select USB as the boot device. Thus allowing the tablet to be booted from this USB flash drive.
I have a Samsung Tab 3 10.1, the device model is GT-P5210 with 4.2.2 OS. My thought is I could connect the micro USB to a USB hub that has one of my bootable flash drives plugged in and boot the tablet from the flash drive.
However, I have searched the forums and wiki for posts on this or related to it but I have not see any.
Can someone point to a post that addresses this?

metaylor said:
I create bootable USB flash drives based on Fedora Linux or UEFI that have special purpose applications on them. I am able to use these flash drives on Windows 8 tablets that have the capability to select USB as the boot device. Thus allowing the tablet to be booted from this USB flash drive.
I have a Samsung Tab 3 10.1, the device model is GT-P5210 with 4.2.2 OS. My thought is I could connect the micro USB to a USB hub that has one of my bootable flash drives plugged in and boot the tablet from the flash drive.
However, I have searched the forums and wiki for posts on this or related to it but I have not see any.
Can someone point to a post that addresses this?
Click to expand...
Click to collapse
The generic boot-loader (*which resides on the chip-set) has no knowledge of USB etc, as the lk (Little Kernel) is more concerned about trapping keystrokes in order to chain-load into recovery or to boot directly into Android environment (When holding Vol+Down key in this instance) - in pseudo-code (this is from the context/aspect of lk, and also, the memory addresses pertaining to how to read the partitions are hard-coded into this lk so it will know how to process the logic!)
The lk kernel is the de-facto standard by Qualcomm for MSM chipsets (Snapdragon) and adopted by manufacturers such as Sony, Motorola, LG, Samsung and can be found in the AOSP source under bootable/bootloader.
if (Is Volume Down key pressed?) then
chain-load kernel from /recovery partition into particular address in memory and jump to it and start execution, in bringing up the recovery environment
else
chain-load kernel from /system partition into particular address in memory and jump to it and start execution in bringing up the Android environment.
end if.
As the kernel within lk is pretty limited, considering that the binary image of the kernel is burned into the chip and therefore no way of modifying it. And also should be mentioned that lk contains the fastboot protocol in preparation for flashing /boot, /recovery, /system and /data partitions. There are two sequences to boot, primary boot and secondary boot as it is:
Primary Boot -> lk (depending on outcome of logic)
Go into Secondary Boot -> /boot or /recovery
Side note: Samsung is fond of the PBL/SBL (Which is Primary Boot Loader and Secondary Boot Loader respectively) in their jargon when it comes to modding. Thing about Samsung, is that, in some handsets, PBL and SBL may be encrypted (Samsung Wave GT-S8500 is one such example, where porting Android to it was nearly impossible to do because of the DRM within the boot loaders which was a nightmare to deal with and made modding it extremely difficult, nonetheless, it is sort of working via an exploit in the FOTA code!)
This is why there are no extra facilities such as OTG functionality or anything else such as serial communications, reading from SDCard, graphics etc as it would make the lk kernel bigger than is intended. In other words, it is the smallest possible size of kernel that is designated to do just the above pseudo-code happen.
Also, another way of looking at it is this, and this is dependent on the Android version - the USB OTG functionality is fully brought up within the Android environment, i.e when the familiar home screen appears, then OTG's functionality is enabled. Unfortunately not the case when looking at it from lk's perspective.
If you're curious, here's the Qualcomm entry on the above lk which is a part of the tiny C source that has ARM assembly included and found in JellyBean's AOSP source in bootable/bootloader/legacy/usbloader/main.c
Source

[INFO] BOOT PROCESS: ANDROID vs. LINUX

NOTE:
I'm not a developer or Android expert. All information provided here is copied from different internet sources and is to the best of my knowledge. I'll not be responsible for any harm to you or your device resulting from this.
1. PC BOOT PROCESS
Before diving into Android boot process, let's have a look at Linux PC first.
Power Button Pressed
Power On Self Test (POST); identify the devices present and to report any problems
BIOS / UEFI
Necessary hardware initialization (keyboard, disk etc.)
Disk (MBR)
DOS Compatibility Region code (optional)
Bootloader
Active/boot partition (Boot sector)
Kernel
Initrd / initramfs (init)
Services/daemons/processes
BIOS / UEFI is the first software code that is hard-coded on board and runs after we press power button. BIOS runs in real (16 bit) mode of processor, thus it can not address more than 2^20 bytes of RAM i.e. routines can't access more than 1 MiB of RAM, which is a strict limitation and a major inconvenience.
When creating partitions, MBR is saved in LBA0, GPT header in LBA1 and primary GPT in LBA2-33, LBA34 (35th) is the first usable sector. Backup or secondary GPT is saved in last 33 LBAs, last usable sector by OS is ( Total LBAs - 33 ). Partitioning software aligns GPT partitions at larger boundaries, e.g. at LBAs that are multiple of 2,048 to align to 1,048,576 bytes (512 bytes * 2048 = 1 MiB) boundaries. So first sector of first partition is LBA 2048 and so on.
When a system boots, driver of a filesystem is to be loaded in RAM in order to use that filesystem, but driver is itself a file, inside some filesystem. It's like a chicken and egg scenario. So the solution is to always load (as a BIOS/UEFI standard) the first sector on the bootable storage (0/0/1 C/H/S in older schemes and LBA0 in newer), which is (legacy or protective) MBR. This communication between BIOS/UEFI and storage media is through commands which are specific to host controller e.g. ATA commands for devices with SATA/AHCI interface on PC.
Master Boot Record (MBR)
1st 512 bytes (1 sector) at the start of 1st valid disk
Bootstrap code (446 bytes) + Partition Table (64 bytes)
Executable code: Bootloader 1st stage scans partition table and finds 1st sector of active partition (or may point towards intermediate stage)
Partition table provides information about active/bootable partition (and all others as well)
Small size of 64 bytes limits the number of maximum (primary) partitions to 4
Since bootloader unable to understand filesystem (inodes etc.) yet, so MBR is itself executable
Last 2 bytes are boot signatures i.e. to find immediately if disk/drive is bootable or not and hence switch to the next
DOS Compatibility Region
This stage is specific to legacy GRUB, GRUB 2 (default bootloader on most of modern Linux ditros) splits this stage to stage 2 and 3
31.5 KiB / 63 sectors next to MBR, contains filesystem utilities
Still loaded by BIOS routines (or bootloader may use it's own drivers)
Required by certain hardware, or if "/boot" partition (sector containing stage 2) is above 1024 cylinder heads of disk, or if using LBA mode
Volume Boot Record (VBR) / Partition Boot Record (PBR)
Sector no. 63 (64th sector) and above may contain Volume Boot Record or Partition BR, very similar to MBR
Also called Volume Boor Sector, it may be the first boot sector on any partition
NTFS saves VBR as metadata file name $Boot at first clusters, which also contains cluster number of file $MFT. $MFT describes all files on the volume; file names, timestamps, stream names, lists of cluster numbers where data streams reside, indexes, security identifiers (SID's), and file attributes like "read only", "compressed", "encrypted", etc.
If disk isn't partitioned, it's the first boot sector of disk
Boot Partition (if exists)
In MBR scheme, a partition can be marked bootable / active using a flag, usually the first partition of disk
Windows stage 1 bootloader reads and loads only the "Active Partition" from MBR Partition Table
Bootsector or VBR/PBR is read by stage 1 or 1.5 (2 or 3 on GRUB2) bootloader which loads stage 2 (4 on GRUB2) or actual bootloader
MBR / VBR Contains:
Jump instruction (first 3 bytes) i.e. "goto boot code" command
Filesystem header
Executable boot code, usually contains jump instruction for next adjacent sector(s) containing stage 2 bootloader
End of sector (similar to boot signature)
Stage 1 or 1.5 (or 3 on GRUB2) bootloader reads the filesystem table (like MFT / FAT) on partition and loads actual bootloader as a regular file
Bootloader (Actual)
Loaded by previous bootloader from the filesystem of same partition
Loads all necessary filesystem drivers (if any further required)
Configuration is read from database e.g. /boot/grub/ on Linux (GRUB) and <"System Reserved" Partition>/Boot/BCD on Windows (BOOTMGR)
Windows:
BCD is binary file, can be read and modified by commandline tool bcdedit.exe or GUI tool EasyBCD
NTLDR on XP simply used C:\ as active partition reading C:\Boot.ini
Linux:
GRUB makes use of modules to offer extra functionality for complex boot processes
It can show a boot menu to user if needed or configured e.g. for multi-booting or in safe/recovery mode or boot from USB/Network etc.
Locates and loads the kernel of desired OS and ramdisk in RAM
If GRUB is unable to handle the kernel of an OS like Windows, it can be configured for CHAINLOADING i.e. read and execute bootsector of the partition containing Windows bootloader
'os-prober' helps 'grub-install' and 'grub-update' finding Windows boot partition (System Reserved) by reading bootloader configuration in that partition
Kernel
1st MB of kernel from same partition (/boot) loaded in RAM by bootlader in read mode, then switch to protected mode (32-bit) and move 1MB ahead clearing 1st MB
Then swith back to real mode and do same with initrd (if it's separate from kernel)
Kernel contain ramfs drivers to read rootfs from initrd and mount it
Initramfs
Contains minimal filesystem and modules (required drivers which aren't carried by kernel) to access real rootfs (hard driver, NFS etc.)
udev or specific scripts load required modules
<ramdisk>/init is usually a script which loads necessary drivers and mounts real rootfs
finally init switch_root's to real rootfs and executes <real rootfs>/sbin/init; sysV (traditional), upstart (Ubuntu's initiative) or systemD (the latest widely accepted)
init > getty (on virtual terminals) > login (program) > motd > login shell > bashrc / bash_profileRead more about LINUX CONSOLE & VIRTUAL TERMINALS
UEFI
UEFI can understand filesystem contrary to BIOS, hence no limitation of MBR code (446 bytes)
Needs an EFI System Partition (ESP), preferrably of minimum 550MB
ESP partition is formatted as FAT32 but can understand other filesystems such as FAT12 (floppy), FAT16, ISO9660 (CD/DVD), UDF etc.
EFI firmware reads directly <ESP_Partition>/EFI/<vendor>/<boot_programs> as configured in boot manager (which disk, which partition, which program)
Boot programs make use of EFI firmware or EFI shell or GUI Boot Manager to load kernel
If boot program is just the disk, (no partition and no program configured), then fallback program <disk>/<ESP partition>/BOOT/BOOTX64.EFI is executed
Secure boot feature verifies signature of boot program before loading
Multi-booting is easy, just read different entry from ESP partition unlike relying on single bootloader to chain load all available OS's
EFISTUB feature of Linux kernel allows booting kernel directly as a boot_program
UEFI works better with GPT than MBR
Must read:
ANDROID PARTITIONS & FILESYSTEMS
2. ANDROID BOOT SEQUENCE
There might be a single or multiple bootloaders (to give directions how to boot). For a typical android device (most common Qualcomm SoC / ARM processor), boot sequence is as follows:
BootROM (like BIOS on PC). It's integrated with SoC.
Processors, bootloaders
POST
SBL
Parallel loading related stuff from different partitions.
Application BootLoader (aboot)
Primary Boot Mode (if no Kernel detected or if bootloader/download mode key combination applied)
Bootloader/Download Mode
Secondary boot
Kernel (hardware detection and populating /sys, /dev/ and /proc directories as the processes start) and initramfs (creating rootfs and other pseudo filesystems on rootfs)
Init (first process with PID "1". It initiates further loading of processes and daemons)
System / OS (ROM)
Recovery (if recovery mode key combination applied. It's a kernel with UI to perform basic troubleshooting operations)
3. BOOTLOADERS
Bootloader(s) facilitate the the initial starting up of device by taking control from SoC, performing necessary checks, loading required components and then hand over the charge of booting to kernel. RAM is detected at first stage to start loading configuration of other hardware (like keypad, display etc.) in it.
There exist(ed) multiple bootloaders which are executed by different processors, on different devices with different (partition) names like RPM (PBL), DBL (Device Boot Loader; CFG_DATA or sbl1), SBL2, SBL3 (QCSBL) and OSBL (Operating System Boot Loader) etc.
In a nutshell, on modern ARM devices (Qualcomm SoC):
BootROM / iROM and PBL
iROM run by CPU0 on power button press, loaded in iRAM (before RAM is initialized)
It may set up RAM and execute PBL in RAM or leave this for SBL. iROM/PBL is hard-coded on SoC, written during CPU production process and it's closed source.
On devices (such as open boards or some tablets) which support booting from multiple sources like eMMC/sdcard/USB/UART/Network like a PC BIOS, there is an extra stage between iROM and PBL:
IBL (Initial BL)
It's also loaded in iRAM. Depending on CPU pin settings (hidden and soldered or exposed for manual switching) informed by iROM, IBL passes boot mode selection to PBL and optionally checks PBL integrity if itself e-signed by iROM.
SBL or XBL (Preloader)
IBL calls SBL from eMMC/SDCard which supports LCD output. SBL initializes the DDR RAM, loads the trusted firmware (TZ) and the RPM firmware if not loaded by BootROM. SBL calls the final bootloader after self testing the device.
Uboot is open-source secondary bootloader for embedded devices. However sources of SBL can also be obtained from Qualcomm.
ABOOT (APPSBL; predecessor of Little Kernel)
ABOOT loads Partition Table, kernel, splash screen (logo) and modem. It's also responsible for charging mode and fastboot mode. Memory addresses in RAM for boot/recovery partitions are hard-coded in aboot.
Other examples of final (i.e. just before kernel) bootloaders are uboot (traditional Linux bootloader for embedded devices) or manufacturers' developed BL's like hboot (used by HTC) and redboot etc.
Manufacturers put their limitations (say of network carrier i.e. SIM lock and others) at this stage. USB protocol isn't enough and communication with bootloader to hack such restrictions require special devices (called Flashing Box or Service Box in common language), even sometimes a protocol like JTAG i.e. talk directly to microprocessor.
As a norm, all of these stage-1,2,3... bootloaders are simply called BOOTLOADER. While on some devices there is no bootloader partition at all and bootloader(s) resides on SoC.
Coming back to the booting process, after initializing boot process, bootloader (if it's locked) checks the integrity of boot.img (normal boot) or recovery.img (recovery boot), loads them in RAM and transfers control to kernel offering it with "phys_initrd_start" address of compressed (cpio, gzipped) initramfs.
4. KERNEL & INITRAMFS
Once the kernel is loaded and extracted in RAM by bootloader along with parameters, kernel starts executing. Kernel is in fact a self-contained (static) executable binary, made up of many object files (.o) linked together at compile time. Once the architecture and CPU are identified, architecture-dependent code is executed as per parameters passed from bootloader. Then arch-independent stage is executed which includes setting up drivers (display, touch etc.), filesystems like rootfs, tmpfs, proc, ext4 etc. and initializing console as well (if configured). Here the kernel-space ends and user-space begins (what they call it).
Kernel extracts compressed initramfs in rootfs (which itself is ramfs or tmpfs) and executes /init binary which subsequently reads its configuration files /init.rc and other /*.rc files written in Android specific init language. With the help of kernel, init mounts pseudo filesystems /sys and /proc and populates /dev directory containing device node files. Then it mounts /system and all other partitions including /data (also decrypts it if encrypted) and sets (SELinux security) policies, system properties and environment variables (PATH, EXTERNAL_STORAGE etc.). Additionally init also look after any hardware changes (ueventd) and started services changes (watchdog) occurring dynamically.
Finally init starts the runtime located on the system partition. One of the major last processes started by init is Zygote (Java virtual machine) which compiles apps to run for specific architecture (mostly arm / arm64).
DEVICE TREE BLOB
Device Tree Blob (DTB) - created by DT Compiler (DTC) from DT Source (DTS) text - is a mapping of hardware components on a board/SoC and usually a part of kernel source.
PC hardware usually support hardware enumeration through ACPI i.e. kernel may enquire (probe) the buses - PCI (internal devices), USB (external devices), SCSI (storage devices), HDMI/DVI/VGA (display devices) etc. - which device is connected to it.
Buses on embedded devices (including Android devices) mostly don't support enumeration (hardware discovery) because there are usually fixed set of devices and no option for a different OS to be loaded on device. Therefore OS needs to be informed of all connected devices and this is done by providing a standard DTB to kernel. DTB is provided by SoC / motherboard vendor and is usually a part of kernel source. During boot process, DTB is loaded by bootloader at boot time and passed to kernel so that it can discover hardware and create node points accordingly.
We can view device tree on Adroid device by:
Code:
~# ls /sys/firmware/devicetree/base
~# ls /proc/device-tree
DTB may live on a separate dtb/odm partition as specified by AOSP (and was the proposed solution for ARM based embedded Linux devices before Android's birth) but that isn't widely practiced. Usually DTB is appended to kernel zImage/Image.gz or placed at second stage inside boot.img.
VERIFIED / SECURE BOOT
Ensuring a chain of trust from Power ON up to loading of kernel is with the domain of SoC vendor (Qualcomm, Intel etc.) and OEM's. Injecting some malicious or harmful code at any point during booting is made harder to the extent of impossibility.
To ensure a secure booting chain, PBL verifies authenticity of SBL which subsequently verifies integrity of bootloaders (TZ, RPM, DSP, HYP and aboot) so that to avoid loading of unsigned images (boot, recovery, system and others). TZ, after being loaded by SBL also verifies ABOOT using a hardware-based root certificate.
A bootloader with Verified/Secure Boot implementation verifies boot.img or recovery.img (kernel, initramfs and DTB appended to kernel or on second stage of boot.img) by matching their signature with key(s) stored in "OEM keystore" (some partition like CMNLIB, KEYMASTER or with some other name) which itself is signed by OEM. Some vendors allow replacing/appending this keystore with custom one so that custom signed images can be flashed followed by re-locking of bootloader. A simple detail is given here.
At this stage, the chain of trust is handed over to "dm-verity" key stored in boot image initramfs, responsible for "Verified Boot" process of Google/AOSP. Dm-verity (a part of Verified Boot implementing Linux Device Mapper by Google) is a kernel feature i.e. it comes into action after boot image (kernel and ramdisk) is loaded in RAM. It verifies subsequently loading block devices; /system, (/vendor if it exists) and optionally others.
For details see this, this and this.
Google suggests integrating libavb (native code to verify integrity of boot.img) in bootloaders starting from Verified Boot 2.
Unlocking Bootloader
Read here to know about the risks of BL unlocking.
Unsigned kernel or recovery cannot be loaded unless bootloader is unlocked. To make any modification to OS, a critical piece of process is disabling a security system built into the Android's bootloader (aboot) that protects the read-only partitions from accidental (or intentional) modification for privacy, security and DRM. This is what's referred to as "unlocking NAND" or "unlocking bootloader." You have to firstly unlock bootloader to modify partitions "boot" or "recovery" and to gain root access on /system. If bootloader is locked, you only have write access to /cache and /data partitions. Everything else is read-only on device and bootloader will prevent unsigned images from being flashed to the phone. Unlocked bootloader ignores signature verification check which was initiated by BootROM and then transferred to "SBL" and then to "ABOOT" while loading kernel or recovery.
Some newer devices don't allow unlocking of bootloader directly (FRP) without permission from manufacturer to ensure more security i.e. contents of partition "devinfo" are signed by the OEM and can't be modified without their approval. After having permission, an official method is provided to unlock BL using PC. Still some functions related to Proprietary Content might be lost due to bootloader unlocking.
DRM is used to protect content from being copied.
Certain pre-loaded content on your device may also be inaccessible due to the removal of DRM security keys.
Click to expand...
Click to collapse
Android Rooting
Must Read: Root User and Linux Capabilities: Linux vs. Android
Note: Unlocking Bootloader and Rooting breaks "Verified Boot". It can be dangerous.
In order to perform some privileged task on Android, we need to "root" the device first. Since it's impossible to start a process with elevated privelages from within running Android OS, rooting usually involves running a root process (su-daemon) from boot with all capabilities. Superuser requests are made by any non-privelaged programs by executing "su" binary and permissions are managed by an app.
In early days, rooting usually involved booting into a custom recovery which in turn mounted and modified /system files. Usually some daemon's executable binary was replaced with a custom script. In order to address the OTA and other issues caused by improving security features (SELinux, Verfied Boot, SafetyNet etc.), systemless root method was introduced which is used by latest apps like Magisk. It involves modifying /boot image and putting some files on /data as well. So a new init service is injected fulfilling all necessary requirements of new security mechanisms.
In both cases, a locked bootloader won't boot custom recovery or modifed kernel (boot.img). See Verified Boot. Therefore bootloader needs to be unlocked for rooting.
However it is possible to gain root sometimes without unlocked bootloader but not always.
Other methods of rooting a phone from within a running ROM using some sort of One-Click rooting solution (KingRoot, Z4Root, KingoRoot etc.) depend on some vulnerability or exploit in Android OS. Making such security breaches is getting harder and harder with every new release of Android and with improved defense mechanisms, though it varies for different vendors too. The most prominent was with the release of Lollipop and Marshmallow when systemless method had to be introduced beacuse the previous methods failed to work. When phone is rooted using one of such improper root methods, there is a high probability to face "incomplete root" like messages at some point. If such a rooting method works for your device, it's alarming. This exploit is also a way for malware to enter your device. For examples, see Magisk Installation - Exploits, this and this. A very popular exploit dirty cow was patched later.
In addition to that, there are some hacks for certain devices to flash custom recovery without unlocking bootloader using some kind of Firmware Flasher tool (SPFlasher, MiFlasher etc.) in Download Mode because Download Mode provides access to device even before bootloader/fastboot is loaded. Or if you are expert in coding, you can mimic the custom recovery image look like the factory signed firmware and flash it through stock recovery. But this exploit isn't a universal solution either.
So the proper way to rooting which doesn't need any vulnerability, goes through unlocked bootloader. While buying a new phone this must be considered. Keeping you away from root access and unlocked bootloader goes in favor of vendors. By forcing you to use their ROMs (with bundle of useless bloatware apps), they earn a lot from you - money as well as forced loyalty - by collecting data, showing ads and using a lot of other tactics. Go for a brand that provides kernel source and ability to unlock bootloader (on customer's responsibility and with voided warranty obviously).
FIRMWARE UPDATE PROTOCOLS (BOOTLOADER MODE)
Likewise BL, on every device there might be a single or multiple BL modes with different names like bootloader mode, download mode, emergency mode (EDL), ODIN (Samsung), nvFlash tool etc. When we boot in BL mode, device is stuck on boot logo. Some factory flashers work in these modes such as MiFlasher (Xiaomi) and SP Flash Tool (for MTK devices). Bootloader or Download Mode is accessible even if device is soft bricked i.e. if Recovery and/or ROM isn't accessible.
Download Mode
Download Mode (certain button combination while powering on device; usually Vol. Up + Vol. Down or Vol. Down for longer duration + Power) is an official method used by many vendors to flash factory firmware / updates using Flasher (software). Emergency Download Mode (EDL), as it's called on Xiaomi Devices, can also be accessed through fastboot/adb commands or by using some jigs/jumpers. However, to ensure more security, EDL is disabled on some newer devices.
Download Mode is primary to bootloader mode (at PBL or SBL stage) and can be used without unlocking bootloader.
Odin (Samsung), QPST/QFIL work in Download mode (Qualcomm HS-USB QDloader 9008).
When we boot in Download mode, device is stuck on blank screen.
Fastboot Mode
Fastboot - provided by ABOOT - is a software development tool and a standard communication protocol for Android bootloader. It's an alternate of recovery flashing that works in BootLoader mode (aboot) and comes bundled on most of the recent ARM Qualcomm devices. It's a minimal UI through commandline to interact with device in case of failure or to modify / flash partitions. Some OEM's provide fastboot with limited functionality e.g. 'fastboot oem' commands not working and some devices haven't at all. It's up to the discretion of mobile phone vendor.
Fastboot mode is used to perform operations through commands when device is connected to PC through USB. It works even when phone is not switched on in Recovery or ROM or even if android isn't installed on phone. You can read here what operations we can perform through fastboot mode.
Only NAND (eMMC) and USB modules (drivers) are activated at this stage.
INIT PROCESSES & SERVICES: ANDROID vs. LINUX
FILESYSTEM TREE MOUNTED BY INIT: ANDROID vs. LINUX
RESOURCES:
From the bootloader to the kernel

RESERVED

RESERVED

RESERVED

RESERVED

You have to firstly unlock bootloader to modify partitions "boot" or "recovery" and to gain root access on /system. If bootloader is locked, you only have write access to /cache and /data partitions. Everything else is read-only on device and bootloader will prevent unsigned images from being flashed to the phone.
Click to expand...
Click to collapse
I'm under the impression that unlocking the bootloader is not mandatory for rooting the device.
You can root the device with a locked bootloader and gain full access to /system partition.

NikosD said:
I'm under the impression that unlocking the bootloader is not mandatory for rooting the device.
You can root the device with a locked bootloader and gain full access to /system partition.
Click to expand...
Click to collapse
Yeah I think my brief statement is a bit misleading because rooting is out of the scope of this thread. I have added some details to first post.

Thank you very much for all this useful info.
Some more comments.
A locked bootloader won't boot custom recovery or modified kernel (boot.img)
Click to expand...
Click to collapse
It happens to have a budget Chinese tablet with Oreo 8.0 and MediaTek SoC, which I can root using a modified/patched boot.img with Magisk v17.1 inside of course - I mean full root without problems - keeping the bootloader locked before and after rooting.
In addition to that, there are some hacks for certain devices to flash custom recovery without unlocking bootloader using some kind of Firmware Flasher tool (SPFlasher, MiFlasher etc.) in Download Mode because Download Mode provides access to device even before bootloader/fastboot is loaded
Click to expand...
Click to collapse
The tablet mentioned above, belongs to this category too.
Using SPFT (Smart Phone Flash Tool), I can flash custom recovery TWRP for my device without unlocking the bootloader.
So, I have two questions:
1) Is it rare to have such a device or is it common nowadays to be able to root and flash custom recovery TWRP with locked bootloader ?
2) How is technically possible to patch boot.img for rooting and flash TWRP using SPFlashTool (even in download mode before bootloader) without complains afterwards from bootloader, verified boot, dm-verity and all these safety checks that validate digital signature of Vendor ?
I mean you can do whatever you want before bootloader starts, but how can you escape from security traps after the initialization of bootloader verifications ?
Thank you.

NikosD said:
1) Is it rare to have such a device or is it common nowadays to be able to root and flash custom recovery TWRP with locked bootloader ?
Click to expand...
Click to collapse
I'm not sure how common it is but I must say these are exploits. Developers are making use of these vulnerabilities for positive and negative purposes. But these are not a "long-term" solution for rooting.
2) How is technically possible to patch boot.img for rooting and flash TWRP using SPFlashTool (even in download mode before bootloader) without complains afterwards from bootloader, verified boot, dm-verity and all these safety checks that validate digital signature of Vendor ?
I mean you can do whatever you want before bootloader starts, but how can you escape from security traps after the initialization of bootloader verifications ?
Click to expand...
Click to collapse
That's what my point is. Fastboot code verifies signatures/hashes only when flashing the image and doesn't verify or fails to verify integrity if image is already flashed. This is not the desired behavior so it's an exploit and it should be closed. Letting unsigned images be flashed in Download Mode is another exploit which is common with Chinese vendors as far as I have come across some instances. They don't address "loopholes" seriously. Failure to stop security breaches at or after bootloader level is definitely on SoC Vendor or OEM's part. I have added a paragraph in first post with some useful details and links.
This link explains:
The Qualcomm SoC is analyzed in the previous chapter dload / edl mode, the mode in the firmware image download process does not do any verification, can be directly written into the brush.
Click to expand...
Click to collapse
It's badly translated from Chinese but is informative.
Exploiting Qualcomm EDL Programmers is a complete series on this subject summarized here.

mirfatif said:
Only NAND (eMMC) and USB modules (drivers) are activated at this stage.
Click to expand...
Click to collapse
Hey pal, I'd like to know if you could help me with an issue I'm facing. I have a Moto G5 that isn't booting to any ROM (it either bootloops in bootlogo or in boot animation), and also on TWRP and during the boot animations the device is slow as hell (like 0.5 FPS on TWRP and even less on boot animation; on TWRP the device also takes a few seconds to complete even the simplest tasks - like the press of a button or the swipe of a slider - here's a video that shows differences between how stuff works on fastboot and how slow things are on TWRP, it takes like 2 hours to completely flash a custom ROM, i.e.).
I know much of the issue will be device-specific, but my point (and the reason I quoted that specific part of your OP) is that, on fastboot mode, the device is snappy and responsive. When I press a button it completes the corresponding task immediately, frames don't stutter (not that there are any animations to be rendered in fastboot, but when I switch from one option to another using the volume keys, it does so on screen as it should, with no lag), and so on. Stock recovery also seems to be ok with speed, but it's even harder to measure than fastboot because, in almost 10 years meddling with android devices, I have always found stock recoveries (and CWM in the pre-TWRP times) to be somewhat slow. Stock recovery definitely looks snappier than TWRP, though. Tried several ROMs, both custom and stock, and the issues remain on all of them.
I got to this post by researching if fastboot mode was stored on the same NAND chip as recovery, OS and so on (found out that yes, it's all on the same chip). If it wasn't, I could just assume it was a hardware fault on the NAND chip, and that would be the reason that fastboot was running fine but recovery and OS weren't, but since they're all on the same cell, I can only think that some part of the system (I mean as in every single code that runs on the device, not only the OS) that loads on TWRP and on normal boot, but not on fastboot (and possibly not on stock recovery) are faulty, thus being a software issue (either solvable with just a normal USB cable or needing a flash box).
So, my question is: which are the differences in the parts of system loaded by fastboot and by TWRP? Are there any parts that are loaded by TWRP that aren't loaded by the stock recoveries on most devices?
I know it's a rather complicated question and some stuff might be device-specific, but if there is anything you could tell me that are more generic to every Android device, it would help me a lot. Thanks in advance.

Question Many questions from a total Android newbie

Hi,
Although I passed my 40 I just bought my first smartphone a few week ago. It’s a Doogee S96Pro. As somebody who like to understand how it works, I already learn a few thing on the Android ecosystem.
I’ve been able to activate the developer mode and to use adb to uninstall some apps. I also managed to boot in fastboot mode to unlock the phone.
My first attempt at flashing was a fail, the phone was rebooting in a loop, indicating "Red state". I guess I should have never "fastboot flash boot/recovery foo.img" if "fastboot boot foo.img" didn’t work? What’s your opinion on this?
The Doogee support sent me a link to the files for my phone. In fact someone on this forum has had already posted it. The two archives are named :
S9S88A7.DGE.DOOGEE.EEA.HB.HJ.AYYDVFAZ.1130.V3.02.zip
S9S88A7.DGE.DOOGEE.HB.HJ.AYYDVFAZ.1203.V3.04.zip
To make the phone boot again I flashed the boot partition with the boot.img file I found in the second archive. If I understood what I read, the file with "EEA" in it’s name is the "European version" while the other one is the "Global version".
Although I flashed with the boot.img that was in S9S88A7.DGE.DOOGEE.HB.HJ.AYYDVFAZ.1203.V3.04.zip, if I go to the update info (About the phone > Update), I can see the string S9S88A7.DGE.DOOGEE.EEA.HB.HJ.AYYDVFAZ.0128.V3.03_20210128-1612. I don’t understand why this difference (v3.03 vs. v3.04).
Then I used the Magisk Manager to patch this boot.img file and flash it again. I now have root access on the phone which is nice.
Now the questions!
When booting the phone says: “Orange state, your phone’s unlocked”, then it boots normally. What’s the implication of this? I’m not sure but I think I tried to lock it again (fastboot flashing lock) but the message remains. Does it sound possible to you? I should check again this point…
In the Magisk Manager I also tried the "SafetyNet" check, which is refused. Is it OK? What does it imply? Why would I need to pass this SafetyNet test for?
I’m not sure I understood how the recovery thing works… I understand it’s another partition than "boot", and I know I can boot on it using the boot menu (pressing volume up when turning on the phone). What I don’t know is if it starts a recovering of the system automatically when booting on this partition (then erasing all data on the phone), or not.
Let’s say I flash the boot.img on the recovery partition (fastboot flash recovery boot.img). If I do a normal boot it should boot as usual, but if I boot on recovery it would boot on a virgin system. Am I right? Let says I configure nothing and reboot again, a normal boot this time. I then should get back to my usual, already configured system, as the "boot" partition hasn’t been modified. Is this also right?
Before doing anymore tests I would like to be able to backup an image with the phone already configured, with data and root access and applications. One (or maybe two or three?) file I can keep on my computer, and in case I break the boot on the phone, I could just fastboot flash boot my_custom_image.img to recover my phone configured. Oy maybe also flashing a "userdata" partition? Would I need some other partition? Is it more complicated than that?
It seems I have to identify the right partition(s) and carefully use dd to dump the partition to an image file… Before trying to do so I’d like to have some advice, hence this post!
Also. I read about a software called TWPR. Should I use it, and why ? I understand it’s a system aimed to be flashed on the recovery partition, is it right? What’s its use?
Finally I read about LineageOS which is the ultimate customization for the phone, it’s a “pure” Android, which is totally opensource (but it has to uses a lot of proprietary blob for devices AFAIK). I don’t think I’ll get there anyway. If I’m not mistaken it’s hard to do, especially with new phones nobody has ran LineageOS on, and there is something like no probability all the devices would work anyway.
Have a nice day.

there's no implication when you see "Orange state, your phone’s unlocked” unless you didn't the one who did it that means your device is tampered ..
also on SafetyNet is broad topic you can learn what it is here https://www.didgeridoohan.com/magisk/MagiskHideSafetyNet also
"Why would I need to pass this SafetyNet test for?" there are multiple reason such as you can't install banking apps,netflix, ...etc nor download them via playstore
moving on it is not recommend to backup userdata partition since it just contain all contains evidence of user activity. It contains call and SMS records, contacts, user-installed apps, app data, settings, and so-on-and-so-forth. In most newer phones, it also is likely to contain photos and videos and other user-generated files unless an external SD card is present. Also it would be impossible to restore userdata partition since android is encrypting it with unique key every time you set up your device https://source.android.com/security/encryption/full-disk
this prevent rooted application crawling on other application data such as paypal just stealing your login info and money
TWRP is like recovery mode but more feature packed (you can backup partition with it not available on stock recovery)
also experience is the best teacher you must experience failure to improve

ineedroot69 said:
Also it would be impossible to restore userdata partition since android is encrypting it with unique key every time you set up your device https://source.android.com/security/encryption/full-disk
this prevent rooted application crawling on other application data such as paypal just stealing your login info and money
Click to expand...
Click to collapse
With a simple ADB command you can decrypt Android partitions:
Code:
adb shell "recovery --set_encrypted_filesystem=on|off" <- enables / diasables encrypted fs

Hi,
Many thx for your answers.
also experience is the best teacher you must experience failure to improve
Click to expand...
Click to collapse
I can confirm that. I accidentally uninstalled the stock launcher with ADB. I’ve been able to install another launcher (I think I’ll keep on Nova Launcher). I tested a few (Launcher<3 and KISS Launcher), although they work fine none of them support switching between running apps. It’s a little bit annoying but I have another way to stop running apps (with App Manager). I guess the only way to get this functionality back is to flash again the boot partition with the Magisk patched image I already used, and to re-configure all the phone again (this is good to learn and luckily I don’t have important data in the phone yet).
Also it would be impossible to restore userdata partition since android is encrypting it with unique key every time you set up your device https://source.android.com/security/encryption/full-disk
Click to expand...
Click to collapse
Couldn’t be possible to dump both boot and userdata partitions and then flash them back both “at once”. The key for encrypting/decrypting the user data being contains in the boot (system ?) partition?
I realize Android has a bunch of security configuration you rarely find on a Linux server. Also the hardware is full of devices which require close-source firmware to operate. This is definitively not a good platform for hacking, like most PC are or a Rasberry Pi is . And I bet Windows and Apple phones are evermore closed…
About encrypting, I have a file called "googlekey/kb_0000000000.bin", which is the same in two archives the support sent me
$ md5sum S9S88A7.DGE.DOOGEE.*/googlekey/kb*
ead8a1d0f11e5f12bdda0f7a22935c2b S9S88A7.DGE.DOOGEE.EEA.HB.HJ.AYYDVFAZ.1130.V3.02/googlekey/kb_0000000000.bin
ead8a1d0f11e5f12bdda0f7a22935c2b S9S88A7.DGE.DOOGEE.HB.HJ.AYYDVFAZ.1203.V3.04/googlekey/kb_0000000000.bin
Click to expand...
Click to collapse
This file is not “per device” knowing every S96Pro users get the same archives. What’s its purpose?
I wonder the same for many files in this archive but I won’t bother you, I’ll make some search.
The one thing I’d like to understand is why the archive is labelled "1203.V3.04" and the system on my phone (after I flashed the boot partition with (a Magisk patched made from) the boot.img in this archive says : "0128.V3.03_20210128". Could it be related to the Magisk patching? (I didn’t check what I had with the stock boot.img). Or I have been downgraded by Google during install?
With a simple ADB command you can decrypt Android partitions:
Click to expand...
Click to collapse
Thx for this. What does it imply to do so? Will the Android system run with this unencrypted data partition? Is there a way to encrypt it again? (With ADB or directly in the phone?)
I’ve seen there are dozens of partitions on a running Android. So far this is what I understood (is this correct?) : There are three important partitions : boot, recovery and userdata. "boot" et "recovery" are the only ones the device can boot on (except booting from an image in fastboot mode using "fastboot boot boot.img"?). Are they some other important partitions this is important to be aware of?
Having a bootable "boot" and a bootable "recovery" partitions, it should be possible to install two different Android OS? I guess this is not possible and the "recovery" partition is dedicated to recovering (ie: reinstall the system) but I don’t understand how and why exactly. The encrypting thing maybe? The system must have a userdata partition and this one can’t be shared between to system…
I think I should buy an older Android smartphone to make all that kind of test, especially knowing I don’t have any other phone I can use for everyday use… Do you have some advice on brands and models which are more friendly with customization of the system?
Apart of ADB and fastboot, what are the other important tools to know about?
For Android development (I mean development of apps for Android), does everyone use an emulator? What’s the best option for such an emulator on Linux?
Have a nice day.

Marotte said:
For Android development (I mean development of apps for Android), does everyone use an emulator? What’s the best option for such an emulator on Linux?
Click to expand...
Click to collapse
My recommendation is GenyMotion for Linux. This emulator requires VirtualBox for Linux gets installed before.
Install GenyMotion
How To Install GenyMotion (Android Emulator) On Linux | 2DayGeek
2daygeek.com Linux Tips, Tricks & News today :- How to Install GenyMotion (Android Emulator) on Ubuntu, Debian, Linux Mint, openSUSE, Arch Linux, Fedora, CentOS, RHEL, Mageia, Manjaro
www.2daygeek.com
DL VirtualBox
Linux_Downloads – Oracle VM VirtualBox
www.virtualbox.org

Marotte said:
Having a bootable "boot" and a bootable "recovery" partitions, it should be possible to install two different Android OS? I guess this is not possible and the "recovery" partition is dedicated to recovering (ie: reinstall the system) but I don’t understand how and why exactly. The encrypting thing maybe?
Click to expand...
Click to collapse
Partitions /boot & /recovery explained:
/boot
This is the partition that enables the phone to boot, as the name suggests. It includes the kernel and the ramdisk. Without this partition, the device will simply not be able to boot.
/recovery
The recovery partition can be considered as an alternative boot partition that lets you boot the device into a recovery console for performing advanced recovery and maintenance operations on it.
That's what you can do from within the recovery console:
Reboot system now
Install ZIP from SD-card
Install ZIP from Sideload
Wipe data / factory reset
Wipe cache partition
Backup and restore

Secure Startup encryption for Samsung Bootloader

Hi !
Is "Secure start-up" encryption for bootloader strong enough to keep all my data safe from the thief? Basically , all the data stored in the Emmc memory is encrypted , so , even if they swap (hot swap) the Emmc chip out of the phone's PCB into a Emmc programmer/reader , they can't read the data without the magic encryption key . It's strong enough this feature to keep the sensitive data , as an example (credit card credentials) , safe against any intruder ? So , there isn't an exploit for that , and the encryption level should be AES128 if I'm not wrong ? It's 100% safe ?

Boot encryption has NOTHING to do with Android's data ( disk-based / file-based ) encryption: Secure boot is a process that validates firmware images on devices before they are allowed to execute, secure boot helps to ensure that only authorized code can execute before the operating system loads.
Means: Secure boot cryptographically validates the digital signature of all boot components and finally the main OS and all components that run in it ( such as drivers and system apps ).

jwoegerbauer said:
Boot encryption has NOTHING to do with Android's data ( disk-based / file-based ) encryption: Secure boot is a process that validates firmware images on devices before they are allowed to execute, secure boot helps to ensure that only authorized code can execute before the operating system loads.
Means: Secure boot cryptographically validates the digital signature of all boot components and finally the main OS and all components that run in it ( such as drivers and system apps ).
Click to expand...
Click to collapse
but Android encryption? Can I leave there my data ? You didn't answer to it ...

I referred to this thread's title.
Android's symmetric disk-based encryption ( Android 5 up to Android 9 ) and/or file-based encrpytion ( Android 10 and higher ) works as follows: 1st step is Android creates an AES-128 key what is called Masterkey - stored in Android's system. 2nd step is Android creates from user PIN / Password an additional AES-128 key what is used to encrypt the Masterkey - also stored in Android's system.
Knowing this I personally don't think you can crack the doubly encrypted key. But who knows ...

FBE vs FDE: Enforce FDE despite FBE

Hello,
the proper section for my question was a bit tricky for me, so I will put it in General Q&A:
I use a device that comes with File based Encryption - namely Samsung Knox.
(But its not a Samsung Knox specific question)
For that I use some AOSP based ROM - namely LineageOS
(Im not too sure if thats some LOS specific question ... )
Now: File based Encryption comes as default.
But I want some Full Disk Encryption for personal preference.
The GUI of my ROM does not allow me to change the encryption method.
Neither can I deactivate FBE in my ROM, nor can I manually put a second layer with FDE ontop.
So: How to enforce FDE?
I guess with building my own ROM it could be achieved - but obviously I look for some easier way, like shell commands to get rid of FBE first and later use regular ROM capabilities to just use the AOSP internal FDE feature.
Anyone with any experience in this field?
Thanks

IMO you had better asked for help here:
Android Q&A, Help & Troubleshooting
This forum is for all of your questions about Android Development and Hacking. If you need help troubleshooting a problem, please be as specific as possible by describing your software configuration, including the ROM, kernel, and any modifications you've done.
forum.xda-developers.com

It's simply not a choice, you have to live with whatever the ROM provides. Of course you can disable encryption and format userdata at wish unencrypted.
May I ask what's the benefit of FDE compared to FBE?
(btw Samsung encryption is unrelated to Samsung Knox)

aIecxs said:
May I ask what's the benefit of FDE compared to FBE?
Click to expand...
Click to collapse
File Based Encryption (FBE)
Note: That's became default with the release of Android 10 in 2019.
Cold Device – contains a stock background image, user data is locked, and needs bruteforce to access.
Hot Device – background image is visible, the camera is accessible, so data collection can be performed on the phone with the proper tools without knowing the passcode.
Full Disc Encryption with Secure Start-Up (FDE)
Note: That's default encryption since Android 6.
Cold Device (Samsung) – must enter the user’s password before the device will even start
Cold Device (LG) – the operating system is not fully booted without the password
Understanding how to differentiate between cold and hot devices while collecting data will help ensure you use the proper tools.
BTW:
If on device USB-Debugging is enabled - most users do so to have an emergency entrance - a hacker can use ADB to easily intrude in device's Android and deduct data even if they are FBE encrypted, whereas on a FDE encrypted device a hacker stands in front of a closed door, IMO, although ADB may be fully functional.

If you use a screen lock on your Android smartphone, full-disk encryption is enabled by default.

jwoegerbauer said:
If on device USB-Debugging is enabled - most users do so to have an emergency entrance - a hacker can use ADB to easily intrude in device's Android and deduct data even if they are FBE encrypted, whereas on a FDE encrypted device a hacker stands in front of a closed door, IMO.
Click to expand...
Click to collapse
Thx but I asked OP.
adbd is running only after userdata is decrypted, as adb_keys is located /data/misc/adb. therefore that would make no difference (btw hacker would need adbkey.pub of victims PC) despites most user don't have enabled usb debugging at all.
There is no difference between FBE and FDE on "Hot Device" after first unlock (decrypt), except that FBE is more secure before first unlock, as the moment android lock screen appears, FDE whole disk is already decrypted, while FBE is splitted in (ce) credential encrypted + (de) device encrypted storage.
the so called secure start-up is optional. most stock FDE encrypted devices are simply encrypted with default_password and are easier to break (I have done many times). In such case it's even possible to bypass screen lock by simply deleting locksettings.db, while the same would for sure destroy FBE and make files unrecoverable.
But even on secure start-up it's easier to bruteforce passphrase online, as the (encrypted) DEK is saved in userspace (crypto-footer) only, which (in theory) allows attacker to backup & restore status quo to cheat gatekeeper timeout and it's even possible to reset failed decrypts counter, while on FBE encryption key is stored in TEE only which makes it impossible to backup (and therefore harder to cheat gatekeeper).
Furthermore, once decrypted, on FDE it's (hard but) possible to recover deleted files, while on FBE it's impossible to recover deleted files per design.
I see no reason to believe FBE is less secure than FDE and don't see the benefit.
btw Samsung devices provide (FBE) "Strong protection" which is successor to (FDE) Secure startup:
-> Settings -> Biometrics and security -> Other security settings -> Strong protection
jwoegerbauer said:
If you use a screen lock on your Android smartphone, full-disk encryption is enabled by default.
Click to expand...
Click to collapse
Completely unrelated to each other. Two cases:
1) I personally prefer unlocked bootloader and use no encryption at all (I have disabled forcefully), but still I am using pin as screen lock. (there is no private data on phone, that's my decision)
2) stock devices are encrypted on first boot, even before user reaches initial setup. user can decide to just swipe with no pin at all, still device itself is encrypted, despites it has no screen lock.
This applies to both FDE and FBE.

aIecxs said:
Thx but I asked OP.
adbd is running only after userdata is decrypted, as adb_keys located /data/misc/adb. therefore that would make no difference (btw hacker would need adbkey.pub of victims PC) despites most user don't have enabled usb debugging at all.
Click to expand...
Click to collapse
Only allowed me to do you often do.
If you could boot a device into recovery mode - what method used ever - then ADB is accessible and fully functional, AFAIK.

jwoegerbauer said:
If you could boot a device into recovery mode - what method used ever - then ADB is accessible and fully functional, AFAIK.
Click to expand...
Click to collapse
Right, if you can boot into custom recovery, which avb/dm-verity prevents on locked bootloader, at least it should
Still it makes no difference to bruteforce FDE or FBE then, assuming TWRP magically bypasses gatekeeper timeout.
btw Cellebrite, isn't that the company that got hacked by signal founder after false claim they could crack signal?
https://arstechnica.com/information...turns-the-tables-on-forensics-firm-cellebrite

Database Info

welcome