Related
Hi All,
Here is my first contribution to the community, hope it is useful!
I've rezzed up a construct to produce a new /data on sd. It makes use of a second partition formatted ext2 or ext3 mounted to /system/sd with /system/sd/data then mounted over /data.
I've included a lengthy readme file which should explain things in greater detail.
Read the readme first to get a list of dirs & files this construct adds to the /data, then copy the files as explained and off you go!
currently my /data "volume" is 2GB and all is lovely!
Things to note:
I'm rom JF1.51 ADP1 CRB43, the sdcard is 8gb class6
The sd /data contains several scripts of my own design which if not needed can be deleted. I include my modified bootdeb.sh script as it mounts to /data/local/mnt/debian vs /data/local/mnt (that is a script I modified for my own needs which you may find useful)
Feel free to adjust any of the scripts according to your needs. Just note that if I update the construct, that your changes may be undone if you use it, so back them up in that scenario.
Also, I include my mount.ak.sh script which make use of a loopfs to accomplish the roughly the same thing, but in a single partition environment (intended as proof of concept).
**There is NO booting into recovery and wiping of the phone for this mod!**
Thanks to JesusFreak for the roms and saurik for the Debian info!
Cheers!
Wow! Nice, Going to try it out. Thanks
is that mean after running data2sd the phone's completely running on sd? and so much faster?
Hi,
To followup on xnycen's question, why would we do this? Is the benefit only in providing more space, or is there also a performance increase (or decrease)?
hate to see the sdcard fail with this change
well...
Yes, after the install you are running complete on sd,
but because of the method used, if you reboot, and take out the sdcard before the boot starts, you will be running as you were before the data2sd aka normal; put the sdcard back in, reboot and you are in data2sd mode again.
This effectively gives you two android run modes: "normal" and data2sd. If the SDCore can not be located the Android will boot in normal mode. You can of course try to keep them sync'd, but if you do the data2sd from a clean slate you can effectively have a "safemode" and if you have an issue with some app, go into this "safemode" and do some investigations or adjustments as the SDCore will mount if you put the sdcard in after the system is in "safemode" but the apps and such will not be loaded until you reboot.
The size expansion is the primary benefit of the construct. As far as speed goes, I haven't checked to see if there are any speed gains in moving /data to the sd... Can someone who's interested check that out and let us know?
Darkstrumn said:
Yes, after the install you are running complete on sd,
but because of the method used, if you reboot, and take out the sdcard before the boot starts, you will be running as you were before the data2sd aka normal; put the sdcard back in, reboot and you are in data2sd mode again.
This effectively gives you two android run modes: "normal" and data2sd. If the SDCore can not be located the Android will boot in normal mode. You can of course try to keep them sync'd, but if you do the data2sd from a clean slate you can effectively have a "safemode" and if you have an issue with some app, go into this "safemode" and do some investigations or adjustments as the SDCore will mount if you put the sdcard in after the system is in "safemode" but the apps and such will not be loaded until you reboot.
The size expansion is the primary benefit of the construct. As far as speed goes, I haven't checked to see if there are any speed gains in moving /data to the sd... Can someone who's interested check that out and let us know?
Click to expand...
Click to collapse
Speed is all I'm interested in If it speeds the device up, why not?
Anyone have anything to report about the speed? I am very interested in this
wow having this "dual boot" seems like an incredible idea... can anyone report on this?
can someone please give a more noob friendly instructions? from what I understand as of now is:
1) boot into recovery and wipe the phone (I'm assuming we can keep whatever ROM we had before)
2) place data2sd.img in root of SDcard (fat32)
3) and this is where I get lost... how exactly do I move the sh file to that location? how do I chmod it? how do I run it? what are the adb commands? thanks!
Hmmnm I thought we weren't doing this because of inheriant security risks?
Not to mention what happens when you mount your fat32 partition....are the apps no longer (as well as your data) available?
NO booting into recovery and wiping of the phone!
Hold up people!
**There is NO booting into recovery and wiping of the phone!**
My bad for not being clear on the wipe instructions!
1) (optional) If you are to wipe it (your choice it is not required), then press menu->settings->SD card & phone storage->scroll to bottom of screen and select "Factory data reset". This will wipe the /data partition only removing all apps and settings. BACKUP YOUR /data dir to sd, First or you will need to down load all your apps again to include paid apps!!
2) place data2sd.img in root of SDcard (fat32)
3) terminal in or adb shell in.
4) cp /sdcard/data2sd.sh to /data/local/bin - to copy the script to your user-space
5) chmod 0750 /data/local/bin/data2sd.sh - to make it executable
6) /data/local/bin/data2sd.sh or data2sd.sh may work to run the script
7) once the install is done it should tell you to reboot. If you had the ddms debugger running, you can observer the log as it is working.
8) done. It may boot a little longer if you clean-slate installed and then restored your apps by copying them back into /data/app and /data/app-private which has the affect of reinstalling everything, and thus the boot will take a while if you have alot of apps like I do. Again you can observe this as the Android boots if you have the debugger running.
thanks a lot man, will give this a shot in a few hours! not at my comp right now
security risks and (u)mounting /sdcard
TheDudeOfLife said:
Hmmnm I thought we weren't doing this because of inheriant security risks?
Not to mention what happens when you mount your fat32 partition....are the apps no longer (as well as your data) available?
Click to expand...
Click to collapse
Not sure why there would be any security issues, the Android is already rooted. If someone can steal the sdcard, they can steal the entire Android, and if the sdcard is left lying about, then yes it can be strolen then too.
The construction of the SDCore assumes all the original permissions, so nothing has access to things it shouldn't save the scripts that use the cache as a backup mount point, but it is given the same permissions as /data, so no worries.
With this construct in place, you can't physically remove the sdcard without first powering off the Android, or rebooting it and removing the sdcard first thing. But while the system is operational, you can mount the sdcard to usb and umount it from usb without issue as long as any mounts to the sdcard are released (loopfs, etc) prior to trying. vold will give 10 tries to umount /sdcard to include attempting to kill the processes with file locks on /sdcard when it tries to umount the volume to attach it to usb.
A a matter of fact, if you observe via teh ddms debugger during boot, the vold service is busy checking the sdcard and mounting it well after the system has already mounted the sdcp2 and begun initializing apps from the sd /data.
So, as long as "sdcard partition2" (sdcp2) remains mounted and all, you can mount the sdcard to the connected Host and back and still have access to your apps and data all you want. I reckon this is how the app2sd and dalvik2sd constructs also perform.
The only thing to really note is that the first partition must be vfat (fat32) and the second either ext2 or ext3 (currently as they support file permissions). vold only cares about the first partition which it expects to be a vfat volume. So as long as p1 is vfat and p2 is posix compliant, we're golden!
The only real question I don't have a good answer for is the speed gains question. My card is a 8GB class6 and things seem fairly responsive. I don't know how much different it would be with a class 4 or 2, or the real speed difference with the built in storage. Honestly it feels a little more responsive, but I haven't really done things that make it feel sluggish.
Oh, I saw the .img and assumed you just mounted the .img from fat32. My bad for not reading all the instructions.
Any problems with apps crashing? I have had a lot of quirky issues in the past with merging the entire partition.
Thanks for the response. I like what you've done here. I'll have to try it out for sure.
TheDudeOfLife said:
Oh, I saw the .img and assumed you just mounted the .img from fat32. My bad for not reading all the instructions.
Any problems with apps crashing? I have had a lot of quirky issues in the past with merging the entire partition.
Thanks for the response. I like what you've done here. I'll have to try it out for sure.
Click to expand...
Click to collapse
No worries; the .img file can be thought of as an install cdrom it is only used the one time during install.
As far as apps crashing, no problems!
During my experimentation building the construct, permission mangling happens if apps are moved out of /data/app, /data/app-private but /data/data and maybe /data/dalvik-cache are not cleaned up prior to copying apps back into their respective app dirs.
What happens is the app dirs are monitored. if you move an apk into the dirs android will install the app. Likewise if you delete an app from those dirs PackageManager will uninstall the app. If /data/data already has the dir for the app, it will use it. if the /data/system/packages.xml and /data/data don't have the same id assigned to that app, you get a permissions\\id conflict; the PackageManager will not manipulate the dir if the id's don't match.
The app /data/local/bin/archiveApps.sh, /data/local/bin/softWIPE.sh and /data/local/bin/restoreApps.sh scripts allow one to backup thier apps to the new /data/app-archive and /data/app-private-archive dirs, clean out /data/data/ /data/dalvik-cache, and restore the apps to their app dirs. As the apps are copied into their dirs, the PackageManager will update /data/system/packages.xml proper and rebuild /data/data and the dalvikVM will build the /dalvik-cahce proper.
Someone made a seemingly nice script (I've not used it yet) called fix_permissions.sh that will parse the /data/system/packages.xml and update the ownership id of the apps /data/data sub dir proper and thus is more scalpel like in operation, but that process doesn't make a backup of the apps; so using both may be the most effective replacing the softWIPE.sh part of the process with the fix_permissions.sh instead.
The only time you should run into issues is if say you have 10 apps installed in teh SDCore /data and 6 in Android /data and then attempt to sync them (because the 6 are different than the 10) by copying\\merging Android /data with SDCore /data the SDCore /data/data /data/dalvik-cache and /data/system/packages.xml will now be mangled.
/data/system/packages.xml will now no longer know about the 10 apps that were installed prior, and on boot PackageManager will "install" them, but /data/data will have dirs inside with now different id's from what was newly assigned them in /data/system/packages.xml, and the 10 apps will now not work correctly but the 6 will as dirs with matching id's were created when they were "installed" new.
To fix, either the softWIPE.sh process outlined above, or the fix_permissions.sh process also mentioned above.
So did anyone try this at all?
Can this be done after doing the apps2sd by marcusmaximus? I tried it but when trying to chmod 0750 /data/local/bin/data2sd.sh it said "No such file or directory" so i tried mkdir /data/local/bin and got "File Exists" so i proceeded with copying data2sd.sh to /data/local/bin again which went without showing any errors and when trying to chmod 0750 /data/local/bin/data2sd.sh again i got the same error so i tried to ls /data/local/bin and just got # what am i doing wrong?
looks interesting! might try this during the weekends ill give an update if i encounter any issues or not
interesting. I'll be looking into this!
I've looked through the big development thread and haven't found a clear answer but has anyone found an app to sd solution that works?
This stupid SdCard / ExtSdCard arrangement is making me pull my hair out. I just realized my titanium backups weren't really backups since they were being stored on my internal memory.
I managed to change my backup folder and free up some space by moving all my backups to the SD card, but for some reason I cannot move apps to SD through Titanium or any other app I've tried.
Does anyone know how to do this? Is there a setting in titanium I need to change or is this just something we have to deal with forever short of flashing a custom rom?
SDCARD binary
Galahad_Knight said:
I've looked through the big development thread and haven't found a clear answer but has anyone found an app to sd solution that works?
This stupid SdCard / ExtSdCard arrangement is making me pull my hair out. I just realized my titanium backups weren't really backups since they were being stored on my internal memory.
I managed to change my backup folder and free up some space by moving all my backups to the SD card, but for some reason I cannot move apps to SD through Titanium or any other app I've tried.
Does anyone know how to do this? Is there a setting in titanium I need to change or is this just something we have to deal with forever short of flashing a custom rom?
Click to expand...
Click to collapse
I've found the real problem, but not the solution.
My Relay runs ICS. It is rooted, and you have to root to even touch any of this. The phone has a binary, /system/bin/sdcard that controls the internal memory and your SDcard. now, I have found that if you are rooted, you can chmod this binary (or use rooted ES File explorer to remove all executable properties) so that it can no longer run, then reboot. then you can run, as superuser "mount -o rw,bind /mnt/extSdCard /mnt/sdcard"
Seems great... Except vfat doesn't like to be bound in such a fashion, and you quickly lose write access and it starts mangling your files.
vfat just doesn't seem to work for us in this case. I've changed mine back, and reformatted my SDcard to be NTFS and mounted with paragon NTFS/HFS, which has completely stabilized my filesystem. only problem is, I can't get this to mount -o bind worth a damn.
I'm toying with the idea of reformatting my SDCard part vfat and either part NTFS or EXT4. maybe I get get the phone to natively recognize ext4 for the sdcard (even if I have to create a script myself)
keypoints I've found in this process:
1. the sdcard binary only allows access to 1023 files at once on either the SDcard, or the memory block it is using as the SDcard. too many pictures in your gallery? so sorry... can't load them all.
2. the physical device for your sdcard is /dev/block/mmcblk1. the partitions are mmcblk1p* (starting with 1)
3. the phone will not allow the sdcard binary to be killed manually. removing the executable permission and then rebooting the phone seems to be the only way to get it to stop.
4. vfat is prone to errors, and android seems to love screwing with the card. getting off vfat seems to be crucial to this process.
As you've gotten no other replies, and this is a gigantic concern for me as well, I'm going to keep working on this myself, in the hopes that I can get the phone to actually use the SDcard the way we hope it can be used.
I am a programmer with experience working on Linux, brand new to Android. Recently I got my first Android phone, a Moto G 4G (XT1045) and installed CM11-20141009-NIGHTLY-peregrine on it. Today I bought a 32GB SDCard and formatted it to ext4 filesystem with AParted.
So it seems like cyanogenmod kernel should have SDCard fix for kitkat built in. Now I have a specific problem. I am trying to switch Titanium Backup to use sdcard1 as backup the location. So I have to browse to that folder.
My sdcard seems to be mounted in two places:
/mnt/media_rw/sdcard1
/storage/sdcard1
but both of them have problems. /storage/sdcard1 is a fuse mount and is read-only, even to root. I can browse there with TitaniumBackup, but then TitaniumBackup can't write any backups. /mnt/media_rw/sdcard1 is writable, and I can make directories on it from adb. But TitaniumBackup cannot browse inside of it, because media_rw is only accessible to user 1023(AID_MEDIA_RW). So I can't point TitaniumBackup to my sdcard!
How to fix this?
colinw1 said:
I am a programmer with experience working on Linux, brand new to Android. Recently I got my first Android phone, a Moto G 4G (XT1045) and installed CM11-20141009-NIGHTLY-peregrine on it. Today I bought a 32GB SDCard and formatted it to ext4 filesystem with AParted.
So it seems like cyanogenmod kernel should have SDCard fix for kitkat built in. Now I have a specific problem. I am trying to switch Titanium Backup to use sdcard1 as backup the location. So I have to browse to that folder.
My sdcard seems to be mounted in two places:
/mnt/media_rw/sdcard1
/storage/sdcard1
but both of them have problems. /storage/sdcard1 is a fuse mount and is read-only, even to root. I can browse there with TitaniumBackup, but then TitaniumBackup can't write any backups. /mnt/media_rw/sdcard1 is writable, and I can make directories on it from adb. But TitaniumBackup cannot browse inside of it, because media_rw is only accessible to user 1023(AID_MEDIA_RW). So I can't point TitaniumBackup to my sdcard!
How to fix this?
Click to expand...
Click to collapse
Hmmm. There have been problems with ext sd cards in cyanogenmod nightlies in the past and it is bound to happen. Not sure.
But try a CM11 snapshot instead of the nightlies as it is more stable with fewer bugs or wait for nightly updates which are very frequent.
Oh and make sure you do a nandroid backup before trying the CM11 Snapshot just in case.
https://download.cyanogenmod.org/?device=falcon&type=snapshot
Cheers!
CuckinFunt said:
Hmmm. There have been problems with ext sd cards in cyanogenmod nightlies in the past and it is bound to happen. Not sure.
But try a CM11 snapshot instead of the nightlies as it is more stable with fewer bugs or wait for nightly updates which are very frequent.
Oh and make sure you do a nandroid backup before trying the CM11 Snapshot just in case.
Cheers!
Click to expand...
Click to collapse
Thanks for the response - it seems it was not a problem with the ROM.
I've been working on this for about 8 hrs since the last post(!!), and I've made a lot of progress. I guess I'll just post it here in case someone comes across this question, though maybe not in scope of my original question.
It turns out my custom rom cyanogenmod (CM) has a fix for the KitKat modification built in. From the command line, as root, as long as I use /mnt/media_rw/sdcard1 to access the external secondary storage (my sdcard), everything works fine. Apps don't seem to mind that /storage/sdcard1 is mostly readonly.
I tested several non-rooted apps (e.g. doubleTwist), and noticed that they were able to write to various locations on my sdcard, always as user:group 1023:1023, ie MEDIA_RW. So the CM fix works. Note that you DO NOT have to install SDFix Apps or anything like that, or modify /etc/permissions/platform.xml. My original goal had been to get TitaniumBackup to start using my sdcard1. What I had to do was, as root, creat /mnt/media_rw/sdcard1/Backup/TitaniumBackup, set the owner and permissions correctly, and then I was able to switch to that directory in Titanium Backup (note, you have to use the Android system back button to go up a level when navigating the directories in Titanium Backup).
Also, It's important if you are using ext4 file system for your sdcard1, that it actually be ext4, not ext2, or CM doesn't recognize it. I used AParted to format it to ext4.
Finally, seems like FolderMount is great app for actually USING your sdcard1, once it's writable. I used FolderMount to redirect sdcard0/DCIM to sdcard1/DCIM, and now my camera is saving to the sdcard.
Sorry for maybe stating so much obvious info, but thought it might help some newcomer like myself - also seems like there's not a really good guide out there for users on how to work with the KitKat restrictions. Seems like with a Mod'd ROM like CM11, maybe nothing has changed(??) I am too new to Android to know the answer.
Hello,
Background story:
Yesterday, YouTube app wasn't working, so I made a Titanium Backup and Nandroid backup. I went to "Factory reset" and assuming Google is sane, and still only reset the internal storage not the emulated SD card, I proceeded with the reset expecting to be met with a tick field to choose whether or not to wipe the SD card. Surprisingly, my expectations weren't met. So turning on my phone afterwards revealed the hard truth... the very hard truth. I lost everything. The damn thing wiped the SD card as well.
Question:
Now to the question, after accepting what happened, I'm now in the process of restoring everything from scratch. Though I see that some apps are storing their data on /storage/emulated/legacy/ directory and others are choosing /storage/emulated/0/... I remember I always used storage/emulated/0 before the accident. So I want to confirm, which of the two is the emulated EXTERNAL SD card? Where do I store my "Data" stuff?
Clarification:
Here's the folder structure:
/storage/
has /storage/emulated/ AND /storage/sdcard0 (shortcut)
/storage/emulated/
has /storage/emulated/0 AND /storage/emulated/legacy
/storage/emulated/0
has /storage/emulated/0/(folders include another 0 and also regular SD card files). Accessing the second /0/ shows clockworkmod/
/storage/emulated/legacy
has has /storage/emulated/legacy/(folders include another 0 and also regular SD card files). Accessing the second /0/ shows clockworkmod/
So legacy and 0 has the same inside folders. Deleting one file also deletes it from the other. So what's the difference?
info:
Nexus 5
KitKat 4.4
Rooted
CWM recovery
They're all links to the same place. Go to general and then "sticky roll-up". In there i wrote a nexus 5 data thread that explains them all
If you have a 0 folder within a 0 folder, that means you had a problem which I discuss briefly in my /sdcard problems thread, also in general
rootSU said:
They're all links to the same place. Go to general and then "sticky roll-up". In there i wrote a nexus 5 data thread that explains them all
If you have a 0 folder within a 0 folder, that means you had a problem which I discuss briefly in my /sdcard problems thread, also in general
Click to expand...
Click to collapse
Thanks, I'll check that out.
UPDATE:
I saw this thread here, and it seems like case 5 is closest to mine. Though the wording is a bit confusing or not an exact replica of my case since I see ALL of my SD card files in the /emulted/0/ and /emulated/legacy/ not just a 0/ folder.
When I open /Data/ I find absolutely nothing! /Storage/emulated/ has everything. Do I move stuff (copy-paste) from /emulated/0/ to /data/media/0 (i.e. I'd have to create media/ and then 0/ myself)? I think I'm wrong, so a bit more help would be much appreciated. Also, stating the final destination of the SD card would help (the desired result) so I can move stuff around to get that result.
devil snake said:
Thanks, I'll check that out.
UPDATE:
I saw this thread here, and it seems like case 5 is closest to mine. Though the wording is a bit confusing or not an exact replica of my case since I see ALL of my SD card files in the /emulted/0/ and /emulated/legacy/ not just a 0/ folder.
When I open /Data/ I find absolutely nothing! /Storage/emulated/ has everything. Do I move stuff (copy-paste) from /emulated/0/ to /data/media/0 (i.e. I'd have to create media/ and then 0/ myself)? I think I'm wrong, so a bit more help would be much appreciated. Also, stating the final destination of the SD card would help (the desired result) so I can move stuff around to get that result.
Click to expand...
Click to collapse
The best thing you could do to get you're device working correctly is to flash the factory image again. Right afterwards do a factory reset. You'll have to resetup your device again but it should work correctly
Sent from my Nexus 9 using XDA Free mobile app
I have a cheapo DragonTouch Y88X. I'm not looking for a custom ROM or anything - its for the kids and honestly it does everything they need, with one exception: Disney Movies Anywhere can't download videos to the external SD card.
As near as I can tell, this is an application problem - hardwired looking for /mnt/sdcard instead of using the Google API to find where the SD card is mounted, and DragonTouch was brain dead an mounted part of the internal storage to /mnt/sdcard (the actual SD card gets mounted to /mnt/extsd).
OK, so I'm not going to get Disney to change the app, so I started poking around how to change the mount points. I'm reasonably familiar with linux, sys admin duties and the like. But the format and options on the fstab file are a bit unfamiliar to me... All I really want to do is make the external SD card be mounted at /mnt/sdcard (and maybe mount the internal partition to something useful - maybe app cache space or something?)
I managed to get root and installed ES File Explorer, and the fstab file (/fstab.sun8i to be specific) has the following lines that I think are relevant...:
...
/devices/virtual/block/nandk auto vfat defaults wait,check,noemulatedsd,voldmanaged=sdcard:auto
/devices/platform/sunxi-mmc.2/mmc_host auto vfat defaults wait,check,noemulatedsd,voldmanagedsd=sdcard:auto
/devices/platform/sunxi-mmc.0/mmc_host /mnt/extsd vfat rw,realtime,fmask=0000,dmask=0000,wait,check,voldmanaged=extsd:auto
/devices/platform/sunxi-mmc.1/mmc_host /mnt/extsd vfat rw,realtime,fmask=0000,dmask=0000,wait,check,voldmanaged=extsd:auto
...
I don't know quite enough about either fstab or Android/DragonTouch's variant of it to know why both sdcard and extsd have two entries each... Or exactly how to swap them. I could try to change the voldmanaged=[name] entry on each - would that be safe?
Is what I'm trying to do crazy, or should this work?
Thanks
Hello,
Welcome to XDA.
Most android devices have a setting to set download location to extsdcard, also, there is usually a setting in most apps that can be used to choose the location the app data downloads to. Since you are rooted then you may be able to create a script that directs your download to where you want but I'm not as knowledgeable about scripts as others here are.
Try posting your question in the forum linked below.
http://forum.xda-developers.com/android/help
The experts there may be able to help.
Remember to register an XDA account so you can post and reply in the forums. Good luck.