Database Info

Workaround: how to use full device encryption with custom recovery and newer ROMs.

Starting with Android L/Lollipop/5.0, full device encryption will be enabled by default, and for good reason. However, we slightly more security-conscious/paranoid SGS3 users have had problems for a while with using custom ROMs and keeping our encryption, as the main custom recovery with support for encryption --- TWRP --- has some incorrect build flags and other problems that aren't considered a priority. However, thanks to the very same bug report and discussion on the issue on TWRP's pages, someone found a solution which I can confirm works for me as well: https://github.com/TeamWin/Team-Win-Recovery-Project/issues/247#issuecomment-52651670
One option would be to check whether CWM Recovery supports your device. Then you'll have access to MiniVold in recovery mode and you can mount encrypted partitions through adb. I prefer the TWR method of just typing in my password, but as long as that does not work on my Galaxy S3, this does:
adb shell
setprop ro.crypto.state encrypted
vdc cryptfs checkpw 'your passphrase here'
mount /dev/block/dm-0 /data
and when you're done backing up/installing a zip
umount /data
Click to expand...
Click to collapse
For Windows users, here's a batch file you can use to automate this process:
Code:
adb shell setprop ro.crypto.state encrypted
adb shell vdc cryptfs checkpw "%~1"
adb shell mount /dev/block/dm-0 /data
Save it under whateveryouwant.bat and then give it the password as a parameter (if the password contains spaces, enclose it in quotes).
Caveat: I've found some operations will unmount /data, which for some reason cannot then be re-mounted by any combination of these commands. Workaround is to just reboot Philz Touch Recovery / CWM Advanced Recovery, re-mount, and continue.

Make system as rw on android 10?

So apparently klaxosaur custom rom doesn't have rw enabled as it is ro locked, is there any way to make it rw permanently? and yes, i have tried @lebigmac 's method on making system rw. on orangefox all it says is "error code 255 please install unzip and try again"

Hi @Fgacko did you try to manually install and run my script as per instructions?
Hopefully I can fix that annoying automatic installation error 255 in next version Not sure why it happens in some recoveries and others not.
Do you have recovery log from /tmp/recovery.txt by any chance?

lebigmac said:
Hi @Fgacko did you try to manually install and run my script as per instructions?
Hopefully I can fix that annoying installation error 255 in next version Not sure why it happens in some recoveries and others not.
Do you have recovery log from /tmp/recovery.txt by any chance?
Click to expand...
Click to collapse
I don't have the recovery log, i didn't try the manual install yet. How do you do it? *I don't understand the instructions for manual installation you said in your thread*

Fgacko said:
I don't have the recovery log, i didn't try the manual install yet. How do you do it? *I don't understand the instructions for manual installation you said in your thread*
Click to expand...
Click to collapse
Alright i've done it, now what?

Fgacko said:
Alright i've done it, now what?
Click to expand...
Click to collapse
Universal init.d enabler still says this

Please send me your log files from this folder:
/data/local/tmp/systemrw_1.32/log/
Thanks.

lebigmac said:
Please send me your log files from this folder:
/data/local/tmp/systemrw_1.32/log/
Thanks.
Click to expand...
Click to collapse
no log file appear there

lebigmac said:
Please send me your log files from this folder:
/data/local/tmp/systemrw_1.32/log/
Thanks.
Click to expand...
Click to collapse
ah i needed to run the script, here
systemrw: Initiating procedure...
systemrw: Device is in Android mode. Ignoring
systemrw: Current device: samsung
systemrw: Current Android version: 10
systemrw: Current SELinux status: Permissive
systemrw: Unable to locate super partition on device. Ignoring
systemrw: /system is already R/W capable. Ignoring
systemrw: /product is already R/W capable. Ignoring
systemrw: /vendor is already R/W capable. Ignoring
systemrw: Adjusting permissions...
systemrw: Attempting to disable dm-verity and verification...
systemrw: There was a problem removing read-only restriction(s) of your device. Abort

lebigmac said:
Please send me your log files from this folder:
/data/local/tmp/systemrw_1.32/log/
Thanks.
Click to expand...
Click to collapse
also, every restart or boot, my tmp folder gets wiped

@Fgacko type this and send me result please:
Bash:
adb shell
su
tune2fs -l /dev/block/by-name/system | grep "features"

lebigmac said:
@Fgacko type this and send me result please:
Bash:
adb shell
su
tune2fs -l /dev/block/by-name/system | grep "features"
Click to expand...
Click to collapse
hmm ok, by any chance you know how to revert build.prop? like the default.prop and prop.default, its cause i accidentally changed a line and its stuck in bootloop

Fgacko said:
hmm ok, by any chance you know how to revert build.prop? like the default.prop and prop.default, its cause i accidentally changed a line and its stuck in bootloop
Click to expand...
Click to collapse
nvm, ill just reflash the rom

Fgacko said:
hmm ok, by any chance you know how to revert build.prop? like the default.prop and prop.default, its cause i accidentally changed a line and its stuck in bootloop
Click to expand...
Click to collapse
That's easy. Simply:
boot into custom recovery
login with your password to decrypt /data
mount /system (/system_root) in the mount options
remount /system (/system_root) as r/w by running mount -o remount,rw /system_root
replace faulty default.prop with your original backup from computer
done

lebigmac said:
That's easy. Simply:
boot into custom recovery
login with your password to decrypt /data
mount /system (/system_root) in the mount options
remount /system (/system_root) as r/w by running mount -o remount,rw /system_root
replace faulty default.prop with your original backup from computer
done
Click to expand...
Click to collapse
problem is, every time i reboot to system it replaces it with the faulty one

Fgacko said:
problem is, every time i reboot to system it replaces it with the faulty one
Click to expand...
Click to collapse
ill just reflash the rom and send you the log file thingy later since its 1:32 am here

lebigmac said:
@Fgacko type this and send me result please:
Bash:
adb shell
su
tune2fs -l /dev/block/by-name/system | grep "features"
Click to expand...
Click to collapse
do i need to run this in adb?

Fgacko said:
do i need to run this in adb?
Click to expand...
Click to collapse
also, when i went to /dev/block/by-name/ i dont see a file or folder called "system", instead it was "SYSTEM" with 0 B

Fgacko said:
also, when i went to /dev/block/by-name/ i dont see a file or folder called "system", instead it was "SYSTEM" with 0 B
Click to expand...
Click to collapse
ok it worked

Thanks for the screenshot. I can't believe you actually got that command to work
It looks like your device doesn't have shared_blocks read-only feature so my script will not be able to remove the read-only lock of your device
If you want you can also check your other partitions such as product or vendor with the same tune2fs -l command but it should be the same result.
You must first try to figure out which Android feature is responsible for the read-only lock of your device.
Please let me know if you figure it out. Thanks.
systemrw: /system is already R/W capable. Ignoring
systemrw: /product is already R/W capable. Ignoring
systemrw: /vendor is already R/W capable. Ignoring
Click to expand...
Click to collapse
According to your script log you can already mount your partitions as R/W. Did you try mounting your partitions as r/w yet (see picture below)?
Are you using custom ROM? Some custom ROM's can do this by default. Sadly not all of them.
Good luck!

lebigmac said:
Thanks for the screenshot. I can't believe you actually got that command to work
It looks like your device doesn't have shared_blocks read-only feature so my script will not be able to remove the read-only lock of your device
If you want you can also check your other partitions such as product or vendor with the same tune2fs -l command but it should be the same result.
You must first try to figure out which Android feature is responsible for the read-only lock of your device.
Please let me know if you figure it out. Thanks.
According to your script log you can already mount your partitions as R/W. Did you try mounting your partitions as r/w yet (see picture below)?
Are you using custom ROM? Some custom ROM's can do this by default. Sadly not all of them.
Good luck!
Click to expand...
Click to collapse
yeah im using klaxosaur custom rom, i tried the command "mount -o remount,rw /" but on every boot it becomes ro again and the command only works for whatever i mounted it. ex if i mounted it in terminal, other apps like mt manager still is ro locked. how do you know which read only lock is on my device?

a7y18lte: Porting on postmarketOS

Hi guys,
Here, for those who are interested, I'm maintaining the packages to use postmarketOS on bare-metal hardware.
I only tested the A750FN variant.
Check it out!
Feedbacks are welcome!
Many thanks:
@VDavid003 for kernel sources

Thanks alot for making this contribution

M0Rf30 said:
Hi guys,
Here, for those who are interested, I'm maintaining the packages to use postmarketOS on bare-metal hardware.
I only tested the A750FN variant.
Check it out!
Feedbacks are welcome!
Many thanks:
@VDavid003 for kernel sources
Click to expand...
Click to collapse
After a week or 2 i can get back home and test it , so keep making these and I'll test them and tell you about my experience

Hello, can you share how you did it? I can't find any files for flashing on the postmarketos wiki page, I have TWRP installed.

Arm1nas said:
Hello, can you share how you did it? I can't find any files for flashing on the postmarketos wiki page, I have TWRP installed.
Click to expand...
Click to collapse
I haven't tested it yet , but you'll have to use the pmbootstrap tool to build a zip that you can flash through the TWRP
The wiki of postmarketOS explains how to use the pmbootstrap tool. Go check it out

animegamer4422 said:
I haven't tested it yet , but you'll have to use the pmbootstrap tool to build a zip that you can flash through the TWRP
The wiki of postmarketOS explains how to use the pmbootstrap tool. Go check it out
Click to expand...
Click to collapse
I figured it out. I created an Ubuntu VM and built it. I am not sure why but now the phone doesn't boot to pmOS, it just spits out some errors or there is only a terminal. TWRP says it can't mount /system, maybe I have to reflash the recovery.

Arm1nas said:
I figured it out. I created an Ubuntu VM and built it. I am not sure why but now the phone doesn't boot to pmOS, it just spits out some errors or there is only a terminal. TWRP says it can't mount /system, maybe I have to reflash the recovery.
Click to expand...
Click to collapse
I don't think it's a issue with the recovery and also it's normal for TWRP to not be able to mount the system
Let's try to diagnose it by going through the error
Could you upload the log and also the screenshot of the error that it spits out
Also upload the built image to some cloud service and send the link here so that i don't have to go through and build the image again
Thanks for testing it out and I'll join you soon so keep trying and I'll try to do everything i can to help you , Have a good day.

animegamer4422 said:
I don't think it's a issue with the recovery and also it's normal for TWRP to not be able to mount the system
Let's try to diagnose it by going through the error
Could you upload the log and also the screenshot of the error that it spits out
Also upload the built image to some cloud service and send the link here so that i don't have to go through and build the image again
Thanks for testing it out and I'll join you soon so keep trying and I'll try to do everything i can to help you , Have a good day.
Click to expand...
Click to collapse
pmos-samsung-a7y18lte.zip
drive.google.com
I think that it's trying to install itself to /system partition which is very small, and the "data" partition is like 50GB. Maybe it's a problem with PBRP (I can't install twrp, it doesn't work for me).
Can you try installing this file and testing if it works for you, or is it the same problem?

Arm1nas said:
pmos-samsung-a7y18lte.zip
drive.google.com
I think that it's trying to install itself to /system partition which is very small, and the "data" partition is like 50GB. Maybe it's a problem with PBRP (I can't install twrp, it doesn't work for me).
Can you try installing this file and testing if it works for you, or is it the same problem?
Click to expand...
Click to collapse
I'd love to test it , but currently I'm not at my home and my A7 is at home (since It's not my daily driver) but I'll test it as soon as I can get back home
Until then
Keep figuring out things
Also I think you can use the TWRP terminal to Change the size of system partition using the resize2fs command i guess
I found this when I looked into it
First to clarify, Resize Partition is different than Resize File System, TWRP provides latter option. It resizes the ext2/3/4 filesystem to the size of partition it's created on. Filesystem size can be checked with df command when mounted, and partition size with blockdev. Just like HDDs or SSDs on PC, Android's eMMC storage is divided into partitions (can be 50+). Some of them have filesystem e.g. the larger ones userdata and system. In simple cases a filesystem occupies a complete physical partition, but it's not always the case, LVM makes it possible to have single filesystem on multiple partitions. Or the opposite what happened in your case as explained here: Sometimes the zip file of: a custom ROM does not have the correct size for the system partition and if the ROM flashes a raw system image, that image will not take up the full block device. Running resize2fs can fix the size Similarly: Sometimes flashing a factory image may flash userdata with an image with a file system that does not take up the full size of the block device So TWRP can expand the filesystem to reclaim wasted space. That's what happened in your case to the /system partition. The actual size of partition was 2.5 GB, but flashed image was of 1.5 GB. Coming to your query: I realized that the system partition is too big (~1.5GB). The current ROM uses only about 40% of the partition so I want to shrink it to make space for cache and data. You want to resize partition, not filesystem. So resize2fs (which TWRP uses at back end) isn't the right choice. As your partition table suggests, you need to delete and recreate partitions 9 to 12. IT'S DANGEROUS AND NOT RECOMMENDED, you may permanently brick your device. All data on these partitions will be lost, so do backup. Also, fdisk isn't proper tool for GPT, it's for MBR partition scheme. Use parted or gdisk instead, but with great care.
So now you can just resize the system partition in case it's size is smaller than the image (not likely in my opinion)

animegamer4422 said:
I'd love to test it , but currently I'm not at my home and my A7 is at home (since It's not my daily driver) but I'll test it as soon as I can get back home
Until then
Keep figuring out things
Also I think you can use the TWRP terminal to Change the size of system partition using the resize2fs command i guess
I found this when I looked into it
First to clarify, Resize Partition is different than Resize File System, TWRP provides latter option. It resizes the ext2/3/4 filesystem to the size of partition it's created on. Filesystem size can be checked with df command when mounted, and partition size with blockdev. Just like HDDs or SSDs on PC, Android's eMMC storage is divided into partitions (can be 50+). Some of them have filesystem e.g. the larger ones userdata and system. In simple cases a filesystem occupies a complete physical partition, but it's not always the case, LVM makes it possible to have single filesystem on multiple partitions. Or the opposite what happened in your case as explained here: Sometimes the zip file of: a custom ROM does not have the correct size for the system partition and if the ROM flashes a raw system image, that image will not take up the full block device. Running resize2fs can fix the size Similarly: Sometimes flashing a factory image may flash userdata with an image with a file system that does not take up the full size of the block device So TWRP can expand the filesystem to reclaim wasted space. That's what happened in your case to the /system partition. The actual size of partition was 2.5 GB, but flashed image was of 1.5 GB. Coming to your query: I realized that the system partition is too big (~1.5GB). The current ROM uses only about 40% of the partition so I want to shrink it to make space for cache and data. You want to resize partition, not filesystem. So resize2fs (which TWRP uses at back end) isn't the right choice. As your partition table suggests, you need to delete and recreate partitions 9 to 12. IT'S DANGEROUS AND NOT RECOMMENDED, you may permanently brick your device. All data on these partitions will be lost, so do backup. Also, fdisk isn't proper tool for GPT, it's for MBR partition scheme. Use parted or gdisk instead, but with great care.
So now you can just resize the system partition in case it's size is smaller than the image (not likely in my opinion)
Click to expand...
Click to collapse
The /system partition is 3.9G big.
This is everything:
https://imgur.com/MOKz0uR

Arm1nas said:
The /system partition is 3.9G big.
This is everything:
https://imgur.com/MOKz0uR
Click to expand...
Click to collapse
Sorry i was sick from past 2 days so couldn't reply earlier
Anyways could you like drop to a shell when you boot up postmarketOS
If yes then check the dmesg
I also think you'll have to flash a special kernel which could also be obtained from the pmbootstrap tool
Also it'd be great if you could record a video of the phone booting up into postmarketOS as that'd help us understand until how far it can reach and where it gets stuck
Thanks alot and Take care

animegamer4422 said:
Sorry i was sick from past 2 days so couldn't reply earlier
Anyways could you like drop to a shell when you boot up postmarketOS
If yes then check the dmesg
I also think you'll have to flash a special kernel which could also be obtained from the pmbootstrap tool
Also it'd be great if you could record a video of the phone booting up into postmarketOS as that'd help us understand until how far it can reach and where it gets stuck
Thanks alot and Take care
Click to expand...
Click to collapse
Hello, did you make any progress with it? I will try installing pmOS again and recording everything.

This is my installation procedure:
On a Ubuntu 20.04 VM I run:
$ pmbootstrap init
I select my device, DE, go through the setup.
Then I type:
$ pmbootstrap install --android-recovery-zip
$ pmbootstrap export
Then on the phone I unmount everything:
https://imgur.com/PlQZ1oB
Go to ADB sideload:
https://imgur.com/xf1FBzQ
https://imgur.com/H9AbkXY
I connect the phone to my computer and use this command to install pmOS:
$ adb sideload pmos.zip
I recorded the installation procedure:
As you can see it boots up the first time, the file system only has 2.6GB free, when it reboots again, it just boots into busybox terminal. When trying to boot up it gives me these errors:
https://imgur.com/qEtwRlx
I logged in and typed:
$ df -H
This is the output:
https://imgur.com/7n61Vm3
If I try to start the desktop environment with $ startx command I get these errors:
https://imgur.com/8xOA1kc
dmesg output:
https://imgur.com/eizBP3m
https://imgur.com/nP8aAaB
This is the phone's partition layout:
https://imgur.com/9xNmaAV
https://i.imgur.com/M0m3GjW.png

D
Arm1nas said:
This is my installation procedure:
On a Ubuntu 20.04 VM I run:
$ pmbootstrap init
I select my device, DE, go through the setup.
Then I type:
$ pmbootstrap install --android-recovery-zip
$ pmbootstrap export
Then on the phone I unmount everything:
https://imgur.com/PlQZ1oB
Go to ADB sideload:
https://imgur.com/xf1FBzQ
https://imgur.com/H9AbkXY
I connect the phone to my computer and use this command to install pmOS:
$ adb sideload pmos.zip
I recorded the installation procedure:
As you can see it boots up the first time, the file system only has 2.6GB free, when it reboots again, it just boots into busybox terminal. When trying to boot up it gives me these errors:
https://imgur.com/qEtwRlx
Click to expand...
Click to collapse
Sorry for the late reply
Anyways i just got home and have the smartphone I'm my hands
So i can too test it out
Also I think we should try switching to Wayland as X seems to be crashing
By using Wayland if it works we'll know that it's not a Xorg issue but if Wayland too fails we'll know that it's a big in the Kernel and the way it handles hardware
Also could you create a Telegram group for us to chat instantly I think that way we can help each other better and then we'll just create a general guide on fixing things ok XDA
(I cannot post links since I didn't started the thread)[/url]

Arm1nas said:
This is my installation procedure:
On a Ubuntu 20.04 VM I run:
$ pmbootstrap init
I select my device, DE, go through the setup.
Then I type:
$ pmbootstrap install --android-recovery-zip
$ pmbootstrap export
Then on the phone I unmount everything:
https://imgur.com/PlQZ1oB
Go to ADB sideload:
https://imgur.com/xf1FBzQ
https://imgur.com/H9AbkXY
I connect the phone to my computer and use this command to install pmOS:
$ adb sideload pmos.zip
I recorded the installation procedure:
As you can see it boots up the first time, the file system only has 2.6GB free, when it reboots again, it just boots into busybox terminal. When trying to boot up it gives me these errors:
https://imgur.com/qEtwRlx
I logged in and typed:
$ df -H
This is the output:
https://imgur.com/7n61Vm3
If I try to start the desktop environment with $ startx command I get these errors:
https://imgur.com/8xOA1kc
dmesg output:
https://imgur.com/eizBP3m
https://imgur.com/nP8aAaB
This is the phone's partition layout:
https://imgur.com/9xNmaAV
https://i.imgur.com/M0m3GjW.png
Click to expand...
Click to collapse
Sorry for the late reply
Anyways ...post links since I didn't started the thread)

Arm1nas said:
This is my installation procedure:
On a Ubuntu 20.04 VM I run:
$ pmbootstrap init
I select my device, DE, go through the setup.
Then I type:
$ pmbootstrap install --android-recovery-zip
$ pmbootstrap export
Then on the phone I unmount everything:
https://imgur.com/PlQZ1oB
Go to ADB sideload:
https://imgur.com/xf1FBzQ
https://imgur.com/H9AbkXY
I connect the phone to my computer and use this command to install pmOS:
$ adb sideload pmos.zip
I recorded the installation procedure:
As you can see it boots up the first time, the file system only has 2.6GB free, when it reboots again, it just boots into busybox terminal. When trying to boot up it gives me these errors:
https://imgur.com/qEtwRlx
I logged in and typed:
$ df -H
This is the output:
https://imgur.com/7n61Vm3
If I try to start the desktop environment with $ startx command I get these errors:
https://imgur.com/8xOA1kc
dmesg output:
https://imgur.com/eizBP3m
https://imgur.com/nP8aAaB
This is the phone's partition layout:
https://imgur.com/9xNmaAV
https://i.imgur.com/M0m3GjW.png
Click to expand...
Click to collapse
I installed plasma desktop and it seems to be working great though there's no on screen keyboard so i had to use a USB OTG to connect a external keyboard and unlock the device and use it And also the performance is not very great it lags alot probably due to the lack of GPU acceleration

animegamer4422 said:
I installed plasma desktop and it seems to be working great though there's no on screen keyboard so i had to use a USB OTG to connect a external keyboard and unlock the device and use it And also the performance is not very great it lags alot probably due to the lack of GPU acceleration
Click to expand...
Click to collapse
What method did you use to flash pmOS? And what recovery do you have? Mind sharing the built pmOS files, so I can flash it myself?

Arm1nas said:
What method did you use to flash pmOS? And what recovery do you have? Mind sharing the built pmOS files, so I can flash it myself?
Click to expand...
Click to collapse
I created a recovery zip using Pmbootstrap and also I set the free space parameter (after running pmbootstrap init) to 2000mb (2gb)
And then i copied that zip file from the temp directory to a Pendrive and connected the pendrive to the phone in recovery mode (PBRP Recovery) and then flashed it
I'll upload the files in a bit
Also I tried using Phosh but it just gets to the Login TTY and then doesn't boots up and i think that's because the device lacks GPU acceleration

Also here's a Telegram group link which I'll delete once you join the group
~REMOVED~

M0Rf30 said:
Hi guys,
Here, for those who are interested, I'm maintaining the packages to use postmarketOS on bare-metal hardware.
I only tested the A750FN variant.
Check it out!
Feedbacks are welcome!
Many thanks:
@VDavid003 for kernel sources
Click to expand...
Click to collapse
hello, are this thread still active?

[HELP] Internal Storage not accessible a.k.a How to clear com.android.providers.media.MediaApplication data

Hi,
Since a few days I'm unable to see any data in my phone via gallery or file manager or any application, and I'm unable to save any data to internal storage. (example: I'm able to use whatsapp for texting, but I can't receive/send any audio/picture and I can't access the old ones).
And I keep getting this error message:"com.android.providers.media.MediaApplication keeps stopping."
I don't know if that's what causing it or if that's just a consequence of not being able to access internal storage.
I can still see all the used storage from settings though.
Phone has unlocked bootloader, but no root/custom rom/recovery.
I saw other Oneplus users started getting the same error these days... (I have a Oneplus 6, but others have different models like Oneplus Nord).
Someone resolved by wiping system and they were able to retrieve storage content and functionality, but they lost apps data, I'm trying to find another way since I have an unlocked bootloader.
An user on stackoverflow suggested I should clean the com.android.providers.media.MediaApplication app data to regenerate the database, but I couldn't figure out how to do it
I tried cleaning media and media.module packages (not media.MediaApplication) from adb but no luck even if successful.
I tried removing "com.android.providers.media" (not media.MediaApplication, I couldn't find that one) folder from data/data with twrp file manager but no luck...
Is there a way to specifically clean "com.android.providers.media.MediaApplication"?
If anyone has any other suggestion on things to try, post it here please.
Thank you in advance

you wrote you already did successfully - obviously that is not the cause. my suspect is broken encryption (you will figure out from recovery.log)

encrypted files look like this +NiZZaTrs3RFzYegpkEk if encryption is not the reason, it could be symlinks messed up. in android enable usb-debugging and check from adb shell.
(repeat this for each result)
Code:
readlink /sdcard
readlink ...
in TWRP recovery check for files like /data/system/uiderrors.txt
/data/log
(don't know exact names) you can adb pull files to PC and inspect with wordpad or Notepad++
last but no least you can capture logcat during boot for further analysis
Code:
adb logcat > logcat.txt
if you're going to factory reset device, install Magisk from TWRP and backup apps to usb flash drive with Migrate-NG beforehand. copy all your pictures + backups to PC
Code:
adb pull -a /sdcard
if you can't install Migrate or can't get usb-otg to work, get a backup to PC at least, I can tell you how to restore single apps from it later (ignore the confirmation request "WARNING adb backup is deprecated... Now Unlock your device and confirm...")
Code:
adb backup --twrp --compress data

alecxs said:
encrypted files look like this +NiZZaTrs3RFzYegpkEk if encryption is not the reason, it could be symlinks messed up. in android enable usb-debugging and check from adb shell.
(repeat this for each result)
Code:
readlink /sdcard
readlink ...
in TWRP recovery check for files like /data/system/uiderrors.txt
/data/log
(don't know exact names) you can adb pull files to PC and inspect with wordpad or Notepad++
last but no least you can capture logcat during boot for further analysis
Code:
adb logcat > logcat.txt
if you're going to factory reset device, install Magisk from TWRP and backup apps to usb flash drive with Migrate-NG beforehand. copy all your pictures + backups to PC
Code:
adb pull -a /sdcard
if you can't install Migrate or can't get usb-otg to work, get a backup to PC at least, I can tell you how to restore single apps from it later (ignore the confirmation request "WARNING adb backup is deprecated... Now Unlock your device and confirm...")
Code:
adb backup --twrp --compress data
Click to expand...
Click to collapse
Hi @alecxs , I really hope you are still active on this forum...
Probably you won't remember this discussion from back in the days (This was the previous part of our exchange if you would like to refresh your memory).
In the end I ended up using my phone in that state for more than one year with the plan of changing phone soon and not have to deal with the wipe and restoration on the same device (if that's not procrastination.... ).
Well, last week I dropped the phone and the screen died. I bought a pixel 7 as a replacement but now I would like to try to save the data of just a few apps. Most important one being Whatsapp, I don't really care about the rest...
So basically:
-the screen is dead but touch is still working and I was able to get scrcpy to work so I can still use it as a "normal' phone from my computer, so that shouldn't be an impediment.
-as previously stated the phone is not able to access internal storage, but I'm able to access it in recovery mode through adb so pulling files should not be an issue.
-I'm able to install and update apks through adb install.
-whatsapp was not able to execute backups during this year since it could not access internal storage, so I have only old ones.
-bootloader is unlocked, I have twrp but no root.
I still need to try your suggestion above, but wanted to ask this before cause maybe you have a different suggestion to extract a single app. Also, I don't really know how migrate works but if it saves the backup on the internal storage I think that's not gonna work cause every app that needs to save something (that isn't data) doesn't work.
Isn't there a way to just copy the data folder of the app on the new phone to transfer data (like what happens for mac apps)?
Thank you in advance

0xCuter said:
-as previously stated the phone is not able to access internal storage, but I'm able to access it in recovery mode through adb so pulling files should not be an issue.
Click to expand...
Click to collapse
yes, if that is true all fine, you can run the Migrate_EMERGENCY_FLASHABLE.zip from recovery and restore app data on other device with Migrate

alecxs said:
yes, if that is true all fine, you can run the Migrate_EMERGENCY_FLASHABLE.zip from recovery and restore app data on other device with Migrate
Click to expand...
Click to collapse
Hey, it's me again... So, I rooted my new phone and I found the emergency_flashable.zip in the migrate thread, now I just need to flash it, but I cannot use scrcpy in recovery mode to see the screen. I guess there is a way to flash the zip from adb without the need to use the screen, but better to ask since I don't want to do stupid stuff...

yes, you can do it from adb. best is MicroSD Card as target directory. copy the zip into the target directory, mount data system vendor product, then install zip. on the first trial it will fail because packages.list needed. but it will create one you can edit. delete unwanted packages before second trial. you can repeat as often you want, existing backups won't created twice.
Code:
adb push Migrate_EMERGENCY_FLASHABLE.zip /sdcard
adb shell
# mount /system_root
# mount /system
# mount /vendor
# mount /product
# mount /external_sd
# mkdir /external_sd/Migrate
# cd /external_sd/Migrate
# mv /sdcard/Migrate_EMERGENCY_FLASHABLE.zip .
# twrp install ./Migrate_EMERGENCY_FLASHABLE.zip
# exit
adb pull /external_sd/Migrate/packages.list
edit the file with editor and push it back
https://twrp.me/faq/openrecoveryscript.html

alecxs said:
yes, you can do it from adb. best is MicroSD Card as target directory. copy the zip into the target directory, mount data system vendor product, then install zip. on the first trial it will fail because packages.list needed. but it will create one you can edit. delete unwanted packages before second trial. you can repeat as often you want, existing backups won't created twice.
Code:
adb push Migrate_EMERGENCY_FLASHABLE.zip /sdcard
adb shell
# mount /system_root
# mount /system
# mount /vendor
# mount /product
# mount /external_sd
# mkdir /external_sd/Migrate
# cd /external_sd/Migrate
# mv /sdcard/Migrate_EMERGENCY_FLASHABLE.zip .
# twrp install ./Migrate_EMERGENCY_FLASHABLE.zip
# exit
adb pull /external_sd/Migrate/packages.list
edit the file with editor and push it back
https://twrp.me/faq/openrecoveryscript.html
Click to expand...
Click to collapse
One thing I forgot to ask: do I need to root my old phone as well or is twrp sufficient?
And to root I would just need to flash Magisk the same way I would flash the emergency flashable.zip right? Or can I just "twrp install [magisk_package]" without going into the shell?

No need to root old phone, TWRP already provides root access for backup purposes. the new phone should be rooted by flashing magisk patched boot.img because the TWRP install method is deprecated. I know I gave only short hints, so don't hesitate to ask for more detailed instructions in case you don't know what to do for some specific step...

alecxs said:
No need to root old phone, TWRP already provides root access for backup purposes. the new phone should be rooted by flashing magisk patched boot.img because the TWRP install method is deprecated. I know I gave only short hints, so don't hesitate to ask for more detailed instructions in case you don't know what to do for some specific step...
Click to expand...
Click to collapse
Thank You... Yes, the new phone is rooted.
One thing that's not needed for the process but I wanted to understand is: Is there a specific reason why we "twrp install" from shell or could in theory be possible to flash the zip file directly from pc storage?

0xCuter said:
or could in theory be possible to flash the zip file directly from pc storage?
Click to expand...
Click to collapse
the reason is, the backup is created locally in the same directory where the zip file is. never tried sideload but I guess it would try to store in /tmp which is RAM.

alecxs said:
yes, you can do it from adb. best is MicroSD Card as target directory. copy the zip into the target directory, mount data system vendor product, then install zip. on the first trial it will fail because packages.list needed. but it will create one you can edit. delete unwanted packages before second trial. you can repeat as often you want, existing backups won't created twice.
Code:
adb push Migrate_EMERGENCY_FLASHABLE.zip /sdcard
adb shell
# mount /system_root
# mount /system
# mount /vendor
# mount /product
# mount /external_sd
# mkdir /external_sd/Migrate
# cd /external_sd/Migrate
# mv /sdcard/Migrate_EMERGENCY_FLASHABLE.zip .
# twrp install ./Migrate_EMERGENCY_FLASHABLE.zip
# exit
adb pull /external_sd/Migrate/packages.list
edit the file with editor and push it back
https://twrp.me/faq/openrecoveryscript.html
Click to expand...
Click to collapse
I'm trying and when I do "mount /system" from the shell I get this:
mount: '/system' not in fstab
Click to expand...
Click to collapse
Is it normal?

it's either /system_root or /system so one of it will fail. I just listed both.

alecxs said:
it's either /system_root or /system so one of it will fail. I just listed both.
Click to expand...
Click to collapse
I suspected that... is it the same for product and external_sd?
Only system_root and vendor went through

/external_sd is the mount point for MicroSD Card, it may have a different name like /sdcard1 or something. if you have enough disk space on /data, no need for external SD.
/product is maybe not needed, just running with /system_root + /vendor is fine.

alecxs said:
/external_sd is the mount point for MicroSD Card, it may have a different name like /sdcard1 or something. if you have enough disk space on /data, no need for external SD.
/product is maybe not needed, just running with /system_root + /vendor is fine.
Click to expand...
Click to collapse
There is no external MicroSD on Oneplus 6... can I put the migrate folder in /sdcard?
Also, I think I should delete some stuff from storage to make space, since I already pulled it on the pc

/sdcard = /data/media can be used. you can split the packages.list file into few parts so it fits in remaining disk space.

alecxs said:
/sdcard = /data/media can be used. you can split the packages.list file into few parts so it fits in remaining disk space.
Click to expand...
Click to collapse
I tried the install and I get this, even though the zip file is in the same folder I'm in (it even autocompletes from terminal)
OnePlus6:/sdcard/Migrate # ls
Migrate_EMERGENCY_FLASHABLE.zip
OnePlus6:/sdcard/Migrate # twrp install Migrate_EMERGENCY_FLASHABLE.zip
Unable to locate zip file 'Migrate_EMERGENCY_FLASHABLE.zip'.
Installing zip file 'Migrate_EMERGENCY_FLASHABLE.zip'
Error installing zip file 'Migrate_EMERGENCY_FLASHABLE.zip'
Done processing script file
Click to expand...
Click to collapse

make sure you give a path to zip file on install, like ./ or full path. it will fail on first execution. check if packages.list was created successfully.
alecxs said:
on the first trial it will fail because packages.list needed. but it will create one you can edit.
Click to expand...
Click to collapse
you can always pull recovery.log afterwards to see what's going on.
Code:
adb pull /tmp/recovery.log

alecxs said:
make sure you give a path to zip file on install, like ./ or full path. it will fail on first execution. check if packages.list was created successfully.
you can always pull recovery.log afterwards to see what's going on.
Code:
adb pull /tmp/recovery.log
[/cod
[/QUOTE]
Click to expand...
Click to collapse
It launched with the full path. I didn't edit the package.list file and I launched it again.
I got a lot of "migrate.sh: failed:..." and just a few "migrate.sh: finish:..." mostly for system apps, but honestly I really only need to backup whatsapp...
...
Click to expand...
Click to collapse
migrate.sh: failed: com.intsig.lic.camscanner
migrate.sh: failed: com.android.theme.icon_pack.ci
migrate.sh: failed: com.google.android.apps.restor
migrate.sh: 397x packages failed / 0x already exist
# MIGRATE_STATUS: backup finished (27/424)
# DONE.
#
# exiting script
#
# # # # # # # # # # # # # # # # # # # # # # # # # #
Updater process ended with ERROR: 1
I:Install took 69 second(s).
Error installing zip file 'sdcard/Migrate/Migrate_EMERGENCY_FLASHABLE.zip'
Done processing script file
Click to expand...
Click to collapse
These are the things that have been backed up:
OnePlus6:/sdcard/Migrate # ls
Backup_android Backup_com.google.android.networkstack
Backup_com.android.bluetooth Backup_com.google.android.packageinstaller
Backup_com.android.cellbroadcastreceiver Backup_com.oneplus.screenshot
Backup_com.android.emergency Backup_com.qualcomm.qti.cne
Backup_com.android.inputdevices Backup_com.qualcomm.qti.poweroffalarm
Backup_com.android.location.fused Backup_com.qualcomm.qti.smq
Backup_com.android.phone Backup_com.qualcomm.qti.uceShimService
Backup_com.android.providers.blockednumber Backup_org.ifaa.aidl.manager
Backup_com.android.providers.settings Backup_se.dirac.acs
Backup_com.android.providers.telephony Backup_vendor.qti.hardware.cacert.server
Backup_com.android.proxyhandler Migrate_EMERGENCY_FLASHABLE.zip
Backup_com.android.server.telecom backup_app_and_data.sh
Backup_com.android.settings busybox
Backup_com.android.shell migrate.log
Backup_com.android.stk migrate.sh
Backup_com.android.systemui packages.list
Backup_com.google.android.ext.shared readme.txt
Click to expand...
Click to collapse

[SOLVED] Help to mount userdata in linux or repair userdata.img

After an Update of Two magisk modules, my Ulefone Armor 11 5G staied stucked on boot logo, I can only enter in recovery or fastboot.
I try to build a TWRP, but it is not able to mount userdata.
I was able to download with the help of mtkclient all the partition on my phone, even userdata , it took 7 hours.
I wanted to load the image in linux but using mount disk imag or using the command sudo mount -o loop userdata.img ~/Armor_11_5G doesn't do anything not even an error message.
I'm wondering if the filesystem was corrupted during the update.
Is it possible to repair the fylesystem like in Windows?
Thanks

did you previously disable encryption and factory reset long time before the modules updates failed?
what do you mean mount doesn't do anything not even an error message? either it give error message or it succeed.

I didn't disable encryption before updating the modules, I already updated this modules many times.
what do you mean mount doesn't do anything not even an error message? either it give error message or it succeed.
Click to expand...
Click to collapse
That is the problem, it doesn't succed and I don't have an error message. The file is 256 Gbyte big, I don't know if it plays a role. I'm using Ubuntu 22

if phone is encrypted that's just 256G garbage. post the output of
Code:
$ parted <file> unit B print

Here are the results of parted
Code:
Error: /home/*****/Public/userdata.bin.img: unrecognised disk label
Model: (file)
Disk /home/osboxes/Public/userdata.bin.img: 249208733696B
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
Thanks

You may try https://www.cgsecurity.org/wiki/TestDisk_Download
I'd be glad for your feedback.

sorry thought it's whole disk, but it's only 232G file therefore parted won't print partition table
does apply to FDE only
assuming this file is dump of single userdata partition, open with HxD editor. if the partition image is not encrypted, you will see lot zeros within first 1024 bytes.
in that case you can check for file system type is ext4 or f2fs.
Code:
$ xxd -l 1080 dump.img | grep 53ef
$ xxd -l 1024 dump.img | grep 1020.f5f2
But most likely the userdata partition is encrypted, therefore no way to recover data offline.
The easiest way not to load Magisk modules is, not to load Magisk. Flashing stock boot.img will solve it.
Beware, in case you disabled encryption beforehand, booting stock boot.img will force encryption. This may take long time without notice.

I never disabled encryption, I don't know if Magisk do it without informing. I used this phone for an year without a problem.
After the update I left the phone on for one night but nothing happens. I tried to reflash the stock boot image, and again a whole night wait, but again nothing happens.
Reading the fstab the file system should be ext4.
The file is the dump of the whole userdata partition of my Ulefone Armor 11 5G.
I did a backup of the whole system before doing any experiment so if the partition table is corrupted maybe if I reflash back the userdata partition with a working partition table I have again access to the datas.

there is no partition table in userdata partition, I just gave you wrong advise. because the phones total storage is 256G, I made wrong assumptions (you can view partition table from file pgpt.bin)
full 1:1 backup is impossible for FBE encryption because encryption keys are stored in TEE. once you factory reset device backup of userdata + metadata becomes useless.
fstab doesn't tell you what file you just have dumped. if you can't find ext4 super magic (#7) it's impossible to loop mount that file (and impossible to decrypt on linux PC)
if you can't fix boot-loop by stock boot.img then it's unrelated to magisk modules. you can however enable adb in default.prop and capture adb logcat during boot-loop for further analysis. you could also inject own script that deletes some files (only DE encrypted files, CE encrypted files requires lock screen credentials aka pin/pattern)

boot this TWRP and get log from adb
Code:
$ fastboot boot recovery.img
$ adb shell twrp decrypt '1234'
$ adb pull /tmp/recovery.log
https://twrp.me/faq/openrecoveryscript.html

full 1:1 backup is impossible for FBE encryption because encryption keys are stored in TEE. once you factory reset device backup of userdata + metadata becomes useless.
Click to expand...
Click to collapse
With mtk client I was able to do the backup of tee1 and tee2 and also of gpt_backup and gpt_main.
$ xxd -l 1080 dump.img | grep 53ef
$ xxd -l 1024 dump.img | grep 1020.f5f2
Click to expand...
Click to collapse
Doesn't produce any results.
I have immediately the command prompt.
you can however enable adb in default.prop
Click to expand...
Click to collapse
How can I do that? Which value should I change in default.prop?

so your "backup" is encrypted. please note Trustonic Kinibi is TEE OS running in secure memory one can't access or backup with mtkclient. the tee partitions in phone storage do not contain any encryption key (none of the partitions does, secure memory is not even a partition). the only crypto related partition is metadata used for keydirectory of metadata encryption (on top of FBE encryption) but it is useless for backup purposes.
yes you can modify default.prop in boot.img, ro.secure=0 should give root access.
https://forum.xda-developers.com/t/...hone-with-broken-screen.2965462/post-85905033
Code:
ro.secure=0
ro.debuggable=1
persist.service.adb.enable=1
in case the default.prop modification is not sufficient, you need additional command to be executed as root.
Code:
# settings put global adb_enabled 1
as you installed magisk, you could use magisk overlay.d/sbin/ for running startup script.
https://forum.xda-developers.com/t/...ithout-losing-your-data.4383255/post-86934375
aIecxs said:
boot this TWRP and get log from adb
Code:
$ fastboot boot recovery.img
$ adb shell twrp decrypt '1234'
$ adb pull /tmp/recovery.log
https://twrp.me/faq/openrecoveryscript.html
Click to expand...
Click to collapse
How about this TWRP? it should be able to decrypt userdata. if decryption failed, provide recovery.log

Until tomorrow I cannot do a logcat and I cannot find my view logcat on my laptop.
I unpacked boot.img with Carliv Image Kitchen and there is no default.prop, that is present in the recovery as prop.default.
Is there a way to backup secure memory of Trustonic?
How about this TWRP? it should be able to decrypt userdata. if decryption failed, provide recovery.log
Click to expand...
Click to collapse
I already tried that version but it cannot decrypt, that' why I'm trying to build my own version of TWRp with the help of the creator of that version of TWRP, but I'm stucked.

oh, you know how to build TWRP with proper FBE + metadata encryption support? have a look at other Oppo devices how they did... good luck.
regarding default.prop in boot.img (it's a symlink to system unfortunately) you can do it the other way
use magisk overlay.d/sbin/
create a boot script that does the thing with resetprop -n <prop_name> <prop_value>
don't use outdated Carliv Image Kitchen! use osm0sis AIK from link above.

oh, you know how to build TWRP with proper FBE + metadata encryption support? have a look at other Oppo devices how they did... good luck.
Click to expand...
Click to collapse
I'm learning.
I try to integrate the decryption service following the suggestion of ADeadTrouser on Github, but the service doesn't want to start and I don't understand why.
I never checked Oppo, I will take a look at them also, thanks for the suggestion.

I think I figured out now the adb logcat at least. hope that helps
https://forum.xda-developers.com/t/accessing-my-phone-with-a-dead-screen.4542763/post-88016019

I tried your script butr nothing happens, the telephone is not listed when I type
Code:
adb devices
and if I type
Code:
adb logcat
I receive the message waiting for device

you might follow the thread

wenyendev said:
You may try https://www.cgsecurity.org/wiki/TestDisk_Download
I'd be glad for your feedback.
Click to expand...
Click to collapse
I run the software on the image and it identify the contents and can read the encrypted and not encrypted part, that means that all the files are there, but I cannot mount in Linux or in TWRP

The fact that I cannot mount in Linux or TWRP the userdata image/partition can be that is corrupted the partition or the file index?
That would also explain why the script for Magisk provided by aIecxs is not able to copy the adb_key from the cache in the data partition.