Database Info

"Stale NFS file handle"

Hi all
I have all my apps and data moved to the sdcard a while back but my phone crashed and I decided to move only the apps to the sdcard.
I'm trying remove the /system/sd/data folder on the sdcard and keeps getting that error message. How to fix it?
Thanks

This should only happen if an NFS export is mounted to a linux box of some sort, and that export has changed or otherwise become inaccessible.

any idea how to fix it?

i have this problem, i'm not able to delete the data folder beacause of that error. :/

your ext2 partition is corrupted. this happens quite a lot if data is moved to the sdcard. boot up linux and run fsck on the sdcard and that should fix it.

cool. I'll give that a try.
Thanks

I'm having this problem too. Can you provide some basic instructions for how to mount and fix under linux? I have an ubuntu vmware image I can boot to on my windows pc and a usb card reader. Will that suffice?

When you plug the card into an ubuntu box it should automatically mount it as the next available drive. You'll have to figure out what device node the card shows up as, unmount it (umount /dev/<insert device name here>), and then run a filesystem check (fsck /dev/<insert device name here>) on the unmounted card. The utility will report various problems about "inodes" which you will want to say yes to fix. Once it has run through the file system should be in a consistent state and ready for use again.
You run the risk of losing stuff written to the card (which is probably corrupt anyways) when you run the fsck so you may want to take a copy of the data first.
On a side note: I am not sure what the default mount options are listed for moving the stuff onto the sd card in the faqs but I suspect it may help prevent corruption to mount the card on android with the sync option. Though, this will definitely slow writes to the card. It would definitely be a bad idea to remove the card while your G1 is running either way.

equid0x said:
On a side note: I am not sure what the default mount options are listed for moving the stuff onto the sd card in the faqs but I suspect it may help prevent corruption to mount the card on android with the sync option. Though, this will definitely slow writes to the card. It would definitely be a bad idea to remove the card while your G1 is running either way.
Click to expand...
Click to collapse
I was thinking of using the sync option, but then I read this at http://linux.die.net/man/8/mount:
sync
All I/O to the file system should be done synchronously. In case of media with limited number of write cycles (e.g. some flash drives) "sync" may cause life-cycle shortening.
Click to expand...
Click to collapse

Busybox seems to have an fsck command built in, but I don't think all the supporting stuff is there. I'd like to have a way to fsck my ext2 partition while on the go and not near my linux box.
I know that you can't fsck without unmounting the partition and of course it would be bad to unmount the partition with apps on it while the phone is running, but I was thinking it would be nice to be able to boot into the recovery console.
I tried this and attempted to do a fsck on /dev/mmcblk0p2 with the fsck in busybox as follows:
Code:
busybox fsck -t ext2 /dev/mmcblk0p2
But the error I got was that fsck.ext2, which is the actual executable that should be used, isn't there.
What would it take to get this onto the system so that I could boot into recovery and do a quick fsck and then reboot back into phone mode?

I was thinking of using the sync option, but then I read this at http://linux.die.net/man/8/mount:
Click to expand...
Click to collapse
Where flash lifetime is concerned... I agree, this will certainly have some impact. However, the amount of wear concerned really depends on the number of write cycles the particular flash you are writing to can handle, and how good any wear leveling in the memory controller is. Modern flash memory will likely last on the order of years even with tons of writing going on. If all you are moving to the card are the apps, that data will likely be written once (or maybe a few times over the life of an app) and only re-read from that point forward. The caching will eventually commit any data in the buffer to "disk" regardless of how much is actually there. The idea is to line up all the writes so they can be done efficiently. Where ext3 is concerned, the commit interval is 5 seconds by default, I am not sure what it was in ext2 but I imagine it is similar. Ext2 is not really a flash optimized filesystem, but it is readily available on basically any linux distro, and is supported on Android. A better fs for flash drives where write cycles are an issue might be something like jffs2 or yaffs.
At any rate, sd cards are cheap. Why not just throw it away when it starts to die?
But the error I got was that fsck.ext2, which is the actual executable that should be used, isn't there.
What would it take to get this onto the system so that I could boot into recovery and do a quick fsck and then reboot back into phone mode?
Click to expand...
Click to collapse
You would have to compile an ARM6 compatible version of fsck and get it onto the recovery partition so you could run it.

just turn off your phone, pull out the sdcard, boot on a linux os and
then in console type :
fsck -p /dev/your_ext2_partition

Is there any way to clear this error message on a windows xp computer?
Maybe use pargon partion manager, but where do I go to fix it in pargon?

equid0x said:
Ext2 is not really a flash optimized filesystem, but it is readily available on basically any linux distro, and is supported on Android. A better fs for flash drives where write cycles are an issue might be something like jffs2 or yaffs.
Click to expand...
Click to collapse
Hmm... Well, we know that yaffs is supported on Android because that's what the onboard storage uses. So I wonder why the tutorial for apps-on-sdcard suggests creating an ext2 partition? Couldn't we create a yaffs partition on the sdcard and use that instead?
Maybe because yaffs isn't as commonly supported in non-Android partitioning tools (which you would have to use to set up the card initially)?
In any case, if there's no real downside to having the partition be yaffs, how can I go about formatting it as such? I don't recall seeing such an option on gParted or anything, but then again I didn't look that carefully.

So can anyone clue me into how I might add yaffs/yaffs2 support to my desktop linux box? I'd like to try using a yaffs2 partition rather than ext2 to see if I get better reliability, but I need to add support for that filesystem first. Can't seem to figure out how to do that quite yet...

You will likely need to create the partition from the terminal using something like fdisk or cfdisk which will allow you to select the proper filesystem ID. The command for creating the filesystem is mkyaffs. The fstab in android will need to be modified to enable mount of this file system at boot. You will need to install all of the yaffs support tools on a linux desktop to get access to the mkyaffs command.
Yaffs is designed to be used directly on NAND or NOR flash memory (not abstracted through the controller built into an SD card) but it may work anyways. I am not very familiar with the specifics of this particular FS. Most of these flash filesystems are designed to provide a bootable root filesystem for an embedded device.
Yaffs kernel support can be built into a custom kernel with the instructions here:
http://yaffs.net/howto-incorporate-yaffs
Its not likely you will find pre-made packages for any of this in a common distro like Ubuntu. So, you will need to know how to compile it all by hand. A good starting point for a lot of linux info is The Linux Documentation Project at:
http://www.tldp.org/
FWIW I have built homebrew linux based routers for dual ISPs, IPSEC VPN and the like using a lightly modified version of CentOS and 4GB CF cards plugged into an ATA adapter. I used EXT2 on these and they were in production use at a small 13 server farm for a couple of years before being replaced with newer equipment without any failures whatsoever.
I have also used CF cards in small 200Mhz cube PCs as basic web kiosks for extended periods of time without any failures as well.
Under normal usage patterns on a mobile device probably does not require a large amount of writes in the grand scheme of things. I'd say it is fairly likely that your card will outlive the device you are using it in regardless of the filesystem in use.
If you are seeing lots of corruption I would suspect a flakey/failing SD card or some other hardware related problem. It definitely pays to buy high quality flash media. I would also suggest not allowing the phone to constantly run dead if you know things are being written to the SD card, since random power failures during a write to flash can permanently damage the media.

Alternative to Partitioning your SD Card?

Is there anyone out there who has simply created a image of an ext2 filesystem rather than physically partitioning the SD card?
I was thinking of creating an appstorage.img file and mounting via the loopback device in order to store apps on the SD card. This would allow me to keep the whole card as FAT32.
So on a linux computer, I'd do:
dd if=/dev/zero of=appstorage.img seek=500999999 bs=1 count=1
mke2fs -F appstorage.img
Then I'd copy the appstorage.img onto the SD card.
Finally I'd have to mount the new filesystem image on the phone....so using the terminal emulator app on the phone itself, I'd do something like:
mkdir /appstorage
mount -o loop appstorage.img /appstorage
mv /data/app/* /appstorage
ln -s /appstorage /data/app/
Not sure if I'd need to load an ext2 module first. If so, I'd want to insert that on bootup of the phone. Would I insert that in one of the rc.init or rc.local files to be run at boot?
Same for the mount command?

The problem is that at boot time, /data/app needs to be available the same time the ext2.ko module of a non-rooted phone becomes available.
JesusFreke added ext2.ko support into somewhere before that point. (I just checked, it's in the system.img flash file; found it in JFs build environment)
In order to use your solution, which many people including me have contemplated, you need to have loop.ko available at that very point in time. That means building your own system.img image file and flashing it with fastboot.
Hope this helps!

Thanks...it does help, although that probably puts it beyond my capabilities. I don't think I'm up to building a custom system.img....maybe it isn't as difficult as I think.
This is similar to what LucidREM did, building a custom mod to the JF build?
Wonder if this would be included as part of the next official Android release from Google/TMobile? Obviously they can't/won't want to partition people's SD cards to expand app storage....seems the filesystem image is the only way to go.

Autarkis said:
The problem is that at boot time, /data/app needs to be available the same time the ext2.ko module of a non-rooted phone becomes available.
JesusFreke added ext2.ko support into somewhere before that point. (I just checked, it's in the system.img flash file; found it in JFs build environment)
In order to use your solution, which many people including me have contemplated, you need to have loop.ko available at that very point in time. That means building your own system.img image file and flashing it with fastboot.
Click to expand...
Click to collapse
Hmm...wait a second here....maybe I don't fully understand.
But looking at the modified init.rc from a thread about moving apps to the SD card, it appears that it is mounting files via loopback and then later directly doing the insmod of the ext2.ko module prior to mounting the ext2 partition on the SD card. Take a look:
Code:
on system
mount cramfs [email protected]/system/modules/modules.cramfs /system/modules ro
mount cramfs [email protected]/system/xbin/xbin.cramfs /system/xbin ro
insmod /system/modules/2.6.25-01843-gfea26b0/kernel/fs/ext2/ext2.ko
mknod /dev/mmcblk0p2 b 179 2 1000
mount ext2 /dev/mmcblk0p2 /system/sd noatime nodiratime
Am I missing something? Are the first two lines utilizing the loopback device already, meaning it would have been loaded previously?

bump. (very curious about this thread)

A bump from me too - last night I had to format my 8Gb SD card back to Fat32 and put myself back to the standard JF RC9 build - my G1 stopped reading my SD card's Fat32 partition and no matter how many times I formatted it or re-created the partition, nothing would work.
Only after FULLY formatting and loosing my my Ext2 partition on it did it start reading it again, causing my phone to fail to boot until I put the standard JF build back on and do a full factory reset.
This is a pretty fundamental flaw of the phone - is there really a technical reason why app's can't be run off a Fat32 SD card?

I am curious to this topic as well...

wont u need to edit a file? so it would auto remount if you reboot the phone?

For those that have installed the Debian distro on their phone, you are doing almost the exact same thing. You create a debian.img file and format it as an ext2 filesystem.
Then you mount it via the loopback device as an ext2 filesystem. But that is after your phone is already booted up in Android and running.
So the question is when the loopback module for the kernel gets loaded. From Autarkis post, he indicates the loading of the loop.ko module is a problem and would require a custom boot image.
But it looks to me like the loopback module is already loaded when init.rc is called...so I'm confused as to why this won't work.
I briefly tried it on the 1GB card that came with the phone, but gave up after a few minutes. Didn't seem to be working. Dunno...maybe I'll give it another go. But I have to believe that someone else has already tried this rather than go through the pain of partitioning.

Wouldn't you then have the same disadvantage as debian.img on FAT? i.e. that you couldn't unmount the FAT (on connection to a PC) as the ext2 partition image was still mounted.
I was thinking about doing the opposite; moving my debian.img over to within the (ext2) app partition to get around this very issue, I can't really work out why you are trying to introduce this problem!

digitalspaghetti said:
A bump from me too - last night I had to format my 8Gb SD card back to Fat32 and put myself back to the standard JF RC9 build - my G1 stopped reading my SD card's Fat32 partition and no matter how many times I formatted it or re-created the partition, nothing would work.
Only after FULLY formatting and loosing my my Ext2 partition on it did it start reading it again, causing my phone to fail to boot until I put the standard JF build back on and do a full factory reset.
This is a pretty fundamental flaw of the phone - is there really a technical reason why app's can't be run off a Fat32 SD card?
Click to expand...
Click to collapse
I think there's a couple reasons the phone can't install apps on a fat32 sd card. The biggest one is that you can't make symlinks to/from a FAT32 file system, meaning even if you were google and wanted to make it possible, it'd likely have to be an all or nothing deal and they didn't want to make the phone unusable/crash without an SD. For the average hacker here, it'd just be too difficult to make the operating system read apps from FAT32.

SDSPLIT repartition your SDcard on phone (w/o a PC)

!!!OBSOLETE!!! 2009-11-26: This script was very useful in the beginning, but is now obsolete. There are superior methods such as sdparted using better tools, and this script likely no longer works properly on modern builds.
As the title says, this script will repartition your sdcard so that you will have a FAT and an EXT2 partition. This is particularly useful for running apps off of the sdcard.
The app is called sdsplit, it has a strong focus on error prevention and recovery. Use at your own risk and please do let me know if you have any problems. As partitioning your card is inherently risky (especially if you are trying to preserve data from it), this script may help backup your data and any system config scripts which it modifies.
Instead of describing all of the features of the script, I will simply post the help message here since I spent a lot of time trying to make it comprehensive. The design of the script tries to make it easy for you to understand where a failure occurs if one does, and to pick up where you may have left off due to the failure after you work around it.
See the HowTo for Dummies for exact steps, but here is the general idea:
* Download sdsplit to your phone and give it execute permissions.
* Run script as root
* Do not run it from a subdirectory of /sdcard
* Run it with as few applications running as possible (to avoid having files open on the sdcard.)
The script logs its output to /data/sdsplit.log, please post the relevant bits from here if you encounter a problem and want help. But first, check out the FAQ for answers to most questions!!!
Code:
# sdsplit --help
Usage(v1.7): sdsplit [options]
'sdsplit' will by default perform all of its default actions in sequence. Specifying a
specific action will tell sdsplit to only perform that action and skip the others. You
may instead prefix an action with 'no' to prevent that specific action from being
performed while all the others are run. For debugging purposes, all output of sdsplit
will be logged (appended) to /data/sdsplit.log
PREREQUISITES:
* This is desgined to work with the Jesus Freke updates (tested on JF1.43ADP/JF1.5).
Here is why you need a JF update: for busybox, DNS, so the /init.rc invokes /system/init.rc,
and so the ext2.ko module is under /system/modules/2.6.27-00392-g8312baf/kernel/fs/ext2.
* If you are making a FAT partition greater than 2GB and have a pre1.5 android
environment, you will likely need a new 'mkdosfs' executable or internet access
from your phone so that sdsplit can download and install it for you. If a capable
mkdosfs is not found somewhere in your path (or where specified by --pmkdosfs),
sdsplit will attempt to download and install it for you from my repo. This executable
came from a Haykuro 1.5 build.
* Additionally, you will need a 'mke2fs' executable for arm-eabi or internet access
from your phone so that sdsplit can download and install it for you. If mke2fs is
not found somewhere in your path (or where specified by --pmke2fs), sdsplit will
attempt to download and install it for you from my repo. I got this executable from
here: http://sites.google.com/site/juanjosec/android
ACTIONS:
--backup|-b Perform a backup of the /sdcard directory to /data/sdcard
--nobackup|-nb (See --pbackup)
--config|-c Configure the system to handle a new ext2 partition on the
--noconfig|-nc sdcard. This involves adding mount options to /system/init.rc and
to /system/etc/mountd.conf along with creating the /system/sd mount
point for the ext2 partition on (see --pext2). Backups of the
original files will be made (.orig). You should use this for
a JF1.5 build.
--partition|-p Wipe the partition table of the sdcard and create two new
--nopartition|-np partitions, first a 500M FAT32 one (see --fatsize) and then
a Liunx one (for EXT2) on the remainder of the sdcard (see --ext2size).
--mkfs|-mk Make the FAT32 (see --fat[16|32]) and EXT2 filesystems. (partitioning
--nomkfs|-nm only allocates the space for them). This requires the mke2fs
exectuable on the G1/ADP (see --pmke2fs).
--restore|-r Restore the original sdcard backup from /data/sdcard back to /sdcard
--norestore|-nr
Along with the above actions there is one additional action if you do not care to
save your original sdcard data:
--nodata|-nd Equivalent to: --nobackup --norestore
OPTIONS:
-h|--help Usage (short) | Help (this)
--forces|-f Force exectution without prompting up front
--forcemkfs|-fm Force even if the mke2fs executable cannot be found and/or
if the mkdosfs executable is not suitable for large
partitions.
--forceumount|-fu Force even if partition already mounted at /system/sd.
Use with care, if you have an already existing EXT2
partition this will delete everything on it!!
--fatsize|-fs SIZE[KM] Set the size of the FAT partition in bytes, K or M.
(defaults to 500M)
--ext2size|-es SIZE[KM] Set the size of the EXT2 partition in bytes, K or M.
(default to the remainder) If you set this, it will not
set the FAT parition to the remainder, it will remain
500M unless you explicitly set it to something else.
The only reason to use this is if you want to leave space
for another partition.
--fat[16|32] Make the FAT partition of type 16 or 32. The default is
32 since 16 does not seem to work with my ADP.
--pbackup|-pb PATH Set the location of the sdcard backup dir.
--pmke2fs|-pme PATH Set the path to the mke2fs executable (if not already in
your PATH.)
--pmkdosfs|-pmd PATH Set the path to the mkdosfs executable (if not already in
your PATH.)
--pext2|-pe PATH Set the mount point for the new ext2 partition (defaults
to /system/sd)
COPYRIGHT: Martin Fick <[email protected]> GNU GPL2 or above
ChangeLog:
v1.7 2009-05-10 (usability improvements)
* Prompt on startup (confirm requested actions), use -f to override.
* Log invocation string to sdsplit.log
* Removed error when sdcard is empty
* If no ext2.ko is found output a message instead of an error.
more...

WOW i don't need the script since i have my card partitioned, but i didn't even know this was possible. i congratulate your efforts and hope that this works for everyone. would certainly help people that have bad computers or USB lower than 2.0 on their computers. i really don't know what to say, i am looking at the script right now and it's about 10 levels above my coding ability. great job on this

I was reluctant to try partitioning my SD card again after two failures, now I'll have to try this tomorrow. Thank you very much, MartinFick!

i was beginning to wonder why this hasn't been done before...
i applaud you in your efforts sir, ill be sure try this tomorrow.
thank you

Has anyone tried this yet? I already have mine set up but I'm dying to know.
This could make LucidREM's job easier too!

MartinFick said:
Well, here is my paltry contribution to the community. I spent entirely too much time on this! I hope it can help at least one person.
Click to expand...
Click to collapse
ok .. so maybe you DID spend entirely too much time but i would hardly call it paltry .. just skimming thru the code .. looks pretty nice .. i'm sure people will be happy to have the assistance

Great work MartinFick!!! I also have a partitioned card, but this is a great addition to the wonderful mods that already exsist in our community. I do hope it works and I'm sure it will help out many.
I refused an offer of $$$ to mod two phones (from root to apps on sd) because I lost the ability to use paragon (trial over), and "customers" not wanting to give their email address. Now that this may take care of that I might start doing it. I'll wait for reviews. Thanks again MartinFick!!!

Wow, I'm very impressed. Nice work.

So if I read this correctly. Using the following command would give me a 1gig EXT2 partion and the rest goes to FAT32?
sdsplit -es 1000[M]
I'm not ready to try it since I don't have my 8gig card yet, but Id be willing to give it a go.
Another question I notice it sets the default of the FAT to 500meg. Shouldn't that be the other way around? I mean if I dump tons of songs, videos and what not you would think that the majority of the card should be FAT to use it opposed to EXT2.

So if I read this correctly. Using the following command would give me a 1gig EXT2 partion and the rest goes to FAT32?
sdsplit -es 1000[M]
Click to expand...
Click to collapse
No, there is no simple way (that I know of) with fdisk to make the frist partition take a remainder of the disk, so that feature is only available to the second partition. Since the first partition (I believe) will need to be FAT for most hacks to work (and my script currently), that makes EXT2 the second one.
So you must figure out how big you want your FAT partition no matter what (default is 500M). The EXT2 partition can then either be the remainder of the disk or a smaller amount (if you later wanted a third partition.) If you have an 8G disk and you want only 1000M for the EXT2 partition, simply use:
Code:
sdsplit -fs 7000M
which will give you a 7G FAT and a remainder of 1G for the EXT2. Make sense? Since this was confusing, I will try to add a note about this.
Another question I notice it sets the default of the FAT to 500meg. Shouldn't that be the other way around? I mean if I dump tons of songs, videos and what not you would think that the majority of the card should be FAT to use it opposed to EXT2.
Click to expand...
Click to collapse
I guess that is probably a personal decision, I am not sure why we even need the FAT at all (except for interoperability with other devices)?
Due to the limitations that I explained in the first section, I could only hardcode a limit on the FAT partition, not the EXT2. And, since I figured that people with a 1G card would probably want at least 500M EXT2, I opted for the 500M FAT default.
If I figure out a clean way to specify the EXT2 size and let FAT be the remainder, I will consider your suggestion. I was basing the fact that FAT should be first on rules that I read in other threads (this is not an inherent disk partitioning limit). If someone could explain to me why (I assume other components in the android system expect this), perhaps this could be worked around? Another solution if FAT absolutely needs to be P1 and EXT2 P2, is to try and simply put P2 first on the partition (fdisk will usually let you do this.) This way, the EXT2 partition could be set to a fixed size and the FAT one could use the remainder (be second on disk,) but stay numbered P1. Anyone have any thoughts on that suggestion?

Awesome. So basically set the FAT and the EXT2 would set its self to the rest. I by no means was questioning why you did it the way you did it. I just didn't have a clue and it makes complete sense now.
Like I said I can't try this the 7th because the cards I ordered do not arrive until then.

Can somebody
Can somebody post a how to execute this coz i downloaded it and can't understand wat to do next

So if we do select 7GB for the first partition, it'll format it FAT32 correct? As FAT can't use that much space.

Bug Fix Release Update
I update the original post with v1.1 which has some bug fixes in it. The partitioning worked fine in the original script, but my repeated testing seems to have masked some bugs in the automounting of the new partition.
Can somebody post a how to execute this coz i downloaded it and can't understand wat to do next
Click to expand...
Click to collapse
While I am more confident about this version (more personal testing), I would be happier if someone who knew how to recover from potential disasters (someone knows how to use recovery mode) and with a basic linux understanding tried it first.
But, to answer your question, download the new one first (v1.1 which I uploaded after you posted your question), then save this file (perhaps remove the .txt extension) to somewhere in your path (/data/local/bin), and give it execute permissions: chmod 777 /data/local/bin/sdsplit and run it specifying the size of the FAT partition that you want (sample for a 1G FAT partition):
Code:
sdsplit -fs 1000M 2>&1 | tee /data/local/tmp/sdsplit.log

So if we do select 7GB for the first partition, it'll format it FAT32 correct? As FAT can't use that much space.
Click to expand...
Click to collapse
Regardless of size, it will format it as FAT 32 unless you specifically tell it to try FAT 16 (which did not seem to work for me)

MartinFick said:
But, to answer your question, download the new one first (v1.1 which I uploaded after you posted your question), then save this file (perhaps remove the .txt extension) to somewhere in your path (/data/local/bin), and give it execute permissions: chmod 777 /data/local/bin/sdsplit and run it specifying the size of the FAT partition that you want (sample for a 1G FAT partition):
Code:
sdsplit -fs 1000M 2>&1 | tee /data/local/tmp/sdsplit.log
Click to expand...
Click to collapse
Hmm, you need to simplify it further for simpletons like myself.
What we need is a step by step (and I mean step by step, i.e. every single thing you need to do to your phone no matter how obvious) guide on what to to from downloading the app, to ending up with a partitioned SD card.
So, e.g, I have an 8gb SD card, I have some data on that card, I wish to make a partition of 1gb for the EXT2 part, and 7gb for the FAT32. I also dont wish to lose my data.
Should I manually copy my data to my computer, then format the SD card and run the app, or is there a built in thing in the app to backup the SD card (on what though?) and allow you to transfer it back after the partition?
Some questions to clarify (which is why the step by step guide for idiots is needed):
1) do you download the app via the app store, and install it on the phone, or do you download the app onto your computer and transfer to your SD card via USB cable?
2) Do you need to change the file (you say you MAY want to remove the .txt. extension, that a yes or a no?)
3) save the file to somewhere in your path? What does that mean? and where exactly?
4) How do you give it 'execute' permissions? By typing in "chmod 777 /data/local/bin/sdsplit", where do you type this in?
5) are all of these instructions carried out in the command / dos type area of the phone, where you can type in these controls? I am assuming so.
These are just some of the things I do not understand (sorry for my ignorance), which is why we need someone to really clarify the whole process so even a monkey can follow it.
I hope someone has the time and inclination, since I am really pissed of with the android since I ran out of space on my internal memory, makes me want to throw the phone away.

Please be step-by-step specific please.

yeah i have the same questions as him ^^^^^
i have an 8 gig sd card. i want to have my EXT2 to have 800 MB and the rest FAT32. how much do i give the FAT32 portion?

First of all, THANK YOU! This is just what many people need. Great job on making it as simple as possible.
I too would second the request to give us a step by step directions on using this (ie press this button and then....(have to say this in Homer Sipmson's voice))
I could most likely figure it out myself but I don't want to take chances, especially since my card is alrealdy fomatted and works, I wouldn't mind changing the size of my partitions at some point but don't want to take risk of losing 110+ apps..
Once more, BIG THANKS for this piece, I'm sure a lot of people will use it.

Martin I just want to say thank you on behalf of the community.
I reset my phone this evening and tested this out on JF rc33. It's brilliant. You've done a great thing here.
I hope you don't mind but I've posted a how to over on the android section of nokia apps at the request of a few users! Of course i linked to this page and gave full credit!

data2sd

Hi All,
Here is my first contribution to the community, hope it is useful!
I've rezzed up a construct to produce a new /data on sd. It makes use of a second partition formatted ext2 or ext3 mounted to /system/sd with /system/sd/data then mounted over /data.
I've included a lengthy readme file which should explain things in greater detail.
Read the readme first to get a list of dirs & files this construct adds to the /data, then copy the files as explained and off you go!
currently my /data "volume" is 2GB and all is lovely!
Things to note:
I'm rom JF1.51 ADP1 CRB43, the sdcard is 8gb class6
The sd /data contains several scripts of my own design which if not needed can be deleted. I include my modified bootdeb.sh script as it mounts to /data/local/mnt/debian vs /data/local/mnt (that is a script I modified for my own needs which you may find useful)
Feel free to adjust any of the scripts according to your needs. Just note that if I update the construct, that your changes may be undone if you use it, so back them up in that scenario.
Also, I include my mount.ak.sh script which make use of a loopfs to accomplish the roughly the same thing, but in a single partition environment (intended as proof of concept).
**There is NO booting into recovery and wiping of the phone for this mod!**
Thanks to JesusFreak for the roms and saurik for the Debian info!
Cheers!

Wow! Nice, Going to try it out. Thanks

is that mean after running data2sd the phone's completely running on sd? and so much faster?

Hi,
To followup on xnycen's question, why would we do this? Is the benefit only in providing more space, or is there also a performance increase (or decrease)?

hate to see the sdcard fail with this change

well...
Yes, after the install you are running complete on sd,
but because of the method used, if you reboot, and take out the sdcard before the boot starts, you will be running as you were before the data2sd aka normal; put the sdcard back in, reboot and you are in data2sd mode again.
This effectively gives you two android run modes: "normal" and data2sd. If the SDCore can not be located the Android will boot in normal mode. You can of course try to keep them sync'd, but if you do the data2sd from a clean slate you can effectively have a "safemode" and if you have an issue with some app, go into this "safemode" and do some investigations or adjustments as the SDCore will mount if you put the sdcard in after the system is in "safemode" but the apps and such will not be loaded until you reboot.
The size expansion is the primary benefit of the construct. As far as speed goes, I haven't checked to see if there are any speed gains in moving /data to the sd... Can someone who's interested check that out and let us know?

Darkstrumn said:
Yes, after the install you are running complete on sd,
but because of the method used, if you reboot, and take out the sdcard before the boot starts, you will be running as you were before the data2sd aka normal; put the sdcard back in, reboot and you are in data2sd mode again.
This effectively gives you two android run modes: "normal" and data2sd. If the SDCore can not be located the Android will boot in normal mode. You can of course try to keep them sync'd, but if you do the data2sd from a clean slate you can effectively have a "safemode" and if you have an issue with some app, go into this "safemode" and do some investigations or adjustments as the SDCore will mount if you put the sdcard in after the system is in "safemode" but the apps and such will not be loaded until you reboot.
The size expansion is the primary benefit of the construct. As far as speed goes, I haven't checked to see if there are any speed gains in moving /data to the sd... Can someone who's interested check that out and let us know?
Click to expand...
Click to collapse
Speed is all I'm interested in If it speeds the device up, why not?

Anyone have anything to report about the speed? I am very interested in this

wow having this "dual boot" seems like an incredible idea... can anyone report on this?

can someone please give a more noob friendly instructions? from what I understand as of now is:
1) boot into recovery and wipe the phone (I'm assuming we can keep whatever ROM we had before)
2) place data2sd.img in root of SDcard (fat32)
3) and this is where I get lost... how exactly do I move the sh file to that location? how do I chmod it? how do I run it? what are the adb commands? thanks!

Hmmnm I thought we weren't doing this because of inheriant security risks?
Not to mention what happens when you mount your fat32 partition....are the apps no longer (as well as your data) available?

NO booting into recovery and wiping of the phone!
Hold up people!
**There is NO booting into recovery and wiping of the phone!**
My bad for not being clear on the wipe instructions!
1) (optional) If you are to wipe it (your choice it is not required), then press menu->settings->SD card & phone storage->scroll to bottom of screen and select "Factory data reset". This will wipe the /data partition only removing all apps and settings. BACKUP YOUR /data dir to sd, First or you will need to down load all your apps again to include paid apps!!
2) place data2sd.img in root of SDcard (fat32)
3) terminal in or adb shell in.
4) cp /sdcard/data2sd.sh to /data/local/bin - to copy the script to your user-space
5) chmod 0750 /data/local/bin/data2sd.sh - to make it executable
6) /data/local/bin/data2sd.sh or data2sd.sh may work to run the script
7) once the install is done it should tell you to reboot. If you had the ddms debugger running, you can observer the log as it is working.
8) done. It may boot a little longer if you clean-slate installed and then restored your apps by copying them back into /data/app and /data/app-private which has the affect of reinstalling everything, and thus the boot will take a while if you have alot of apps like I do. Again you can observe this as the Android boots if you have the debugger running.

thanks a lot man, will give this a shot in a few hours! not at my comp right now

security risks and (u)mounting /sdcard
TheDudeOfLife said:
Hmmnm I thought we weren't doing this because of inheriant security risks?
Not to mention what happens when you mount your fat32 partition....are the apps no longer (as well as your data) available?
Click to expand...
Click to collapse
Not sure why there would be any security issues, the Android is already rooted. If someone can steal the sdcard, they can steal the entire Android, and if the sdcard is left lying about, then yes it can be strolen then too.
The construction of the SDCore assumes all the original permissions, so nothing has access to things it shouldn't save the scripts that use the cache as a backup mount point, but it is given the same permissions as /data, so no worries.
With this construct in place, you can't physically remove the sdcard without first powering off the Android, or rebooting it and removing the sdcard first thing. But while the system is operational, you can mount the sdcard to usb and umount it from usb without issue as long as any mounts to the sdcard are released (loopfs, etc) prior to trying. vold will give 10 tries to umount /sdcard to include attempting to kill the processes with file locks on /sdcard when it tries to umount the volume to attach it to usb.
A a matter of fact, if you observe via teh ddms debugger during boot, the vold service is busy checking the sdcard and mounting it well after the system has already mounted the sdcp2 and begun initializing apps from the sd /data.
So, as long as "sdcard partition2" (sdcp2) remains mounted and all, you can mount the sdcard to the connected Host and back and still have access to your apps and data all you want. I reckon this is how the app2sd and dalvik2sd constructs also perform.
The only thing to really note is that the first partition must be vfat (fat32) and the second either ext2 or ext3 (currently as they support file permissions). vold only cares about the first partition which it expects to be a vfat volume. So as long as p1 is vfat and p2 is posix compliant, we're golden!
The only real question I don't have a good answer for is the speed gains question. My card is a 8GB class6 and things seem fairly responsive. I don't know how much different it would be with a class 4 or 2, or the real speed difference with the built in storage. Honestly it feels a little more responsive, but I haven't really done things that make it feel sluggish.

Oh, I saw the .img and assumed you just mounted the .img from fat32. My bad for not reading all the instructions.
Any problems with apps crashing? I have had a lot of quirky issues in the past with merging the entire partition.
Thanks for the response. I like what you've done here. I'll have to try it out for sure.

TheDudeOfLife said:
Oh, I saw the .img and assumed you just mounted the .img from fat32. My bad for not reading all the instructions.
Any problems with apps crashing? I have had a lot of quirky issues in the past with merging the entire partition.
Thanks for the response. I like what you've done here. I'll have to try it out for sure.
Click to expand...
Click to collapse
No worries; the .img file can be thought of as an install cdrom it is only used the one time during install.
As far as apps crashing, no problems!
During my experimentation building the construct, permission mangling happens if apps are moved out of /data/app, /data/app-private but /data/data and maybe /data/dalvik-cache are not cleaned up prior to copying apps back into their respective app dirs.
What happens is the app dirs are monitored. if you move an apk into the dirs android will install the app. Likewise if you delete an app from those dirs PackageManager will uninstall the app. If /data/data already has the dir for the app, it will use it. if the /data/system/packages.xml and /data/data don't have the same id assigned to that app, you get a permissions\\id conflict; the PackageManager will not manipulate the dir if the id's don't match.
The app /data/local/bin/archiveApps.sh, /data/local/bin/softWIPE.sh and /data/local/bin/restoreApps.sh scripts allow one to backup thier apps to the new /data/app-archive and /data/app-private-archive dirs, clean out /data/data/ /data/dalvik-cache, and restore the apps to their app dirs. As the apps are copied into their dirs, the PackageManager will update /data/system/packages.xml proper and rebuild /data/data and the dalvikVM will build the /dalvik-cahce proper.
Someone made a seemingly nice script (I've not used it yet) called fix_permissions.sh that will parse the /data/system/packages.xml and update the ownership id of the apps /data/data sub dir proper and thus is more scalpel like in operation, but that process doesn't make a backup of the apps; so using both may be the most effective replacing the softWIPE.sh part of the process with the fix_permissions.sh instead.
The only time you should run into issues is if say you have 10 apps installed in teh SDCore /data and 6 in Android /data and then attempt to sync them (because the 6 are different than the 10) by copying\\merging Android /data with SDCore /data the SDCore /data/data /data/dalvik-cache and /data/system/packages.xml will now be mangled.
/data/system/packages.xml will now no longer know about the 10 apps that were installed prior, and on boot PackageManager will "install" them, but /data/data will have dirs inside with now different id's from what was newly assigned them in /data/system/packages.xml, and the 10 apps will now not work correctly but the 6 will as dirs with matching id's were created when they were "installed" new.
To fix, either the softWIPE.sh process outlined above, or the fix_permissions.sh process also mentioned above.

So did anyone try this at all?

Can this be done after doing the apps2sd by marcusmaximus? I tried it but when trying to chmod 0750 /data/local/bin/data2sd.sh it said "No such file or directory" so i tried mkdir /data/local/bin and got "File Exists" so i proceeded with copying data2sd.sh to /data/local/bin again which went without showing any errors and when trying to chmod 0750 /data/local/bin/data2sd.sh again i got the same error so i tried to ls /data/local/bin and just got # what am i doing wrong?

looks interesting! might try this during the weekends ill give an update if i encounter any issues or not

interesting. I'll be looking into this!

howto apps2sd as ext4

Ok. So lots of talk about ext for apps2sd and how it's being mounted as ext2 even if you formatted to ext4 etc.
Also talk about how journalling hurts performance and such. We'll tackle all those points here.
1. Howto fix the mounting of your ext partition in the meantime till your mod updates their software.
A. Pre-requisites
You must have already partitioned your sd card with an ext4 fs for apps2sd. You can do this in recovery or some other app.
You should also get the program file expert. It's awesome. But any filemanager that has root ability will work. You can do all this in terminal too so whatever floats your boat. I'll use file expert below.
B. Remount your system fs as read/write (not sure if it's actually needed since /system is mounted rw by most mods already... but it can't hurt). You can do this in file expert by hitting menu and selecting More, then the top option labeled "Mount", select read write. It should say success.
You can also do this in terminal but like i said above, i dont think this is absolutely needed but i always assumed it was so never tried this not doing it.
C. go back down directories using the back key till you hit /. Then goto /system/etc/init.d/
D. select S05mountsd (the S0# isn't important here so if yours is different dont worry). Once selected, hit menu and select file operations. Select permissions and change permissions to read/write all.
E. Open the file with a text editor of choice. Scroll down the file to the part that has the command with -t auto in it. It should be pretty obvious, and it's near the end.
Change -t auto to -t ext4. It doesn't matter if your fs is actually ext2 or ext3, mounting as ext4 is always better. The ext4 block allocator and general code is more efficient even if you dont use ext4 features.
You may also want to add an option to the options list data=writeback. This disables journal caching of actual data and only caches metadata. This is really integral to performance on such a slow medium like sd cards and we'll discuss that later.
Save your file and exit.
F. Reboot your phone. You should now have your sd-ext mounted as ext4.
2. data=writeback and why?
there are multiple modes of journalling available to ext4. The default is ordered. This means that there is a specific order that information is written to the fs to ensure that nothing is lost that is said to be written. This means metadata about the fs and file data is both cached in the journal. Safe but slow.
data=writeback still allows crash protection but means you may lose actual newly written data. Your fs wont be corrupt, but your file may not have gotten those last updates you were writing and had not synced yet. It's a pretty decent compromise.
There are other forms that offer better performance but basically remove journal integrity and we're not concerned with those.
There are also options that determine the interval of journal syncing and you can find that information on ext4 man pages.
Why do we care so much about performance here? because sd flash is slow and finite. A journal file is like swap in that it is a static file that's contents get written to anytime you write to your fs. On a normal hdd, it's not noticeable really but on a class 6 SD card, you are going to notice having to write everything twice to your sd card. So we choose writeback mode to remove that double writing and only cache fs metadata updates so that the integrity of the filesystem isn't hurt, and only that particular file that we crashed prior to syncing gets effected. 99% of the time this is fine and perfectly safe and way better than nothing.
3. So if journalling can hurt performance by using up write bandwidth to a medium we are writing other stuff on, wouldn't swap be bad too?
Yes. Swap should only be put on sd cards if you absolutely need it or your phone wont work. Aside from stressing the card, the bandwidth to read/write is extremely small for something you basically have to be interactive with and move to and from ram. Additionally android complicates matters by managing memory at a higher level than the kernel and this means that swap is ignored as a portion of memory. So while technically userspace apps deal with swap transparently, the low memory manager of android will operate as if you didn't have swap...which defeats the purpose.
Use swap as a last choice.
Also, the ability to make swap partitions and swap files is a common issue in linux-land. It's best to use a partition as this removes fs overhead but it becomes even more of a concern when you put it on an ext4 fs with journalling active. With default settings, a swap file will be considered just another file and the journal will cache the data and metadata updates (very bad for performance). if you use swap on an ext4 fs that's actually mounted as ext4 you _NEED_ to make the data option writeback. You will be very disappointed if you dont.
4. Other important notes:
If you use any ext4 specific options like extents, you will no longer be able to read the files written by recovery programs that have not been updated to mount fs's as ext4. Also, just enabling extents doesn't convert data on the fs to use them. You have to write the files while the option is enabled to see their benefit. This is more easily done by putting the card in an actual linux computer and mounting with the correct options and copying the data from the filesystem and then back again ....not for the novice. Though any new files written will automatically have extents even if you dont redo all current data. So best to bother with that after a fresh install prior to restoring apps.
You may also be thinking... Why do i even need a fat fs, i can make the whole thing ext4 and it will be awesome! You would be right, it would be awesome. But it's not so easy. A couple things have to be done in order to make everything written world rewritable and accessible on such a partition. you would have to setgid to a group like "sdcard" to a directory in this new fs and give it rwx group permissions and then have all users current and future become part of that group and mount bind the directory to /sdcard. Needless to say, recovery apps would have to also be updated. Then you could make your main sd partition any fs your kernel supports.

Nice post, cm7 and diet gb I build already have -t ext3 set, do you think ext4 would make much difference?

ext4 has a better block allocator and other features both backwards compatible and ext4 specific. There's really no reason to not mount ext2/3 and 4 as ext4

Thanks for the answer, This is also in the kernel I compile,
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT23=y
CONFIG_EXT4_FS_XATTR=y

Yes, i began my work on the kernel tonight and noticed that in the latest kernels at least that ext4 is being masqueraded as ext2 and as i thought early on as might be the case, that the reporting of the partition as being mounted as ext2 is a bug that has been squashed in the newest vanilla kernels.
This is still an issue, but not a serious one. FS repair tools and such may bug out when trying to found out what fs they're dealing with as the kernel reports the wrong ext version but the functionality of the fs is not affected ( you still get journalling and all that jazz).
So this is less of a performance issue (cept the adding of data=writeback) and more of an issue with other programs needing to see the correct fs.

Ur Diet 09.05 still did not see ext4 partition by Titanium.
carz12 said:
Thanks for the answer, This is also in the kernel I compile,
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT23=y
CONFIG_EXT4_FS_XATTR=y
Click to expand...
Click to collapse
Sent from my Gingerbread on Sapphire using XDA Premium App

If the kernel you're using is one of the newer kernels (probably anything after 2.6.32...but you should be using something much newer) then it likely doesn't even have ext2 or ext3 compiled in. It's all just ext4 like it should be. The issue you still may have however is that the kernel reports the lower revision extfs because ext4's driver accepts the ext4 fs but the kernel is checking for ext2....
Basically, we are asking if this is an ext2fs. The ext4fs driver answers cuz it's all there is and it's masquerading as ext2 and ext3 and ext4. So it looks and sees ext4 and says, hey, i can read this and returns YES. Now the answer yes was to the question of ext2, so that's what gets reported everywhere, but internally, ext4 knows it's got an ext4 fs. mount options that normally would error out with -t ext2 are accepted because the fs driver isn't really ext2 and it knows the fs isn't either so the mount options pass.
This is a kernel bug in an aesthetic and practical sense. Functionally it's all working as intended but other apps and users will get confused as to what exactly is the filesystem, ext2, ext3 or ext4. It can't know without probing the fs itself. Which is stupid for many applications.
Supposedly the latest kernels have a patch to fix this to correctly report the correct fs revision of ext. Though perhaps that hasn't been pushed to linus's branch yet.
In any case, you still should be modifying the mount options to use data=writeback so you're not clogging up the little SD bandwidth you have with data caching in the journal. You really only need fs metadata caching so you dont bork the entire partition on a crash. data=writeback does that, but default is data=ordered.

thank you for such a good post!