Database Info

[DEV] /cache converted to swap

So as we all know /cache is used for OTA updates which, when you have your phone rooted is useless.
What I'm wondering is why we can't use /dev/block/mtdblock4 (aka /cache) as a permanent onboard swap partition. Granted it will only give us an additional 30MB but the speed of this swap will be higher than that of our sdcard.
In order to test this we'll have to:
convert this partition to a linux-swap partition.
mkswap /dev/block/mtdblock4
update init.rc to no longer mount this as yaffs2
add this new swap space to the swap script
The issue is of course being able to run a partition utility on the phone. Second, if this doesn't work we don't have a way to format /cache back to yaffs2 (that I know of).
If you have experience with any of the issues I listed above please comment/post. If you don't have a clue please don't post so we can keep this thread clean. I would appreciate only experienced devs responding.
If this is possible it would benefit Hero/MotoBlur ROMS which are memory intensive.

For your second concern, why would we need to change it back to /cache? Just make it a disclaimer when people attempt this operation.
More importantly, I think the primary issue is that this is rom space is it not? Doesn't this portion of the memory have a limited amount of times that it could be written to?

Hmm why not trying "parted /dev/block/mtdblock4" in recovery? ;-)
rest would be easy.

Couldn't we use mkyaffs2img (in recovery /sbin) to turn it back into yaffs?

persiansown said:
For your second concern, why would we need to change it back to /cache? Just make it a disclaimer when people attempt this operation.
More importantly, I think the primary issue is that this is rom space is it not? Doesn't this portion of the memory have a limited amount of times that it could be written to?
Click to expand...
Click to collapse
Well my concern was if there was some adverse effect of removing /cache from the system we'd need a way to revert back.
I am not aware of any MAX amount of times that /cache can be written to.
*Edited*
Due to unintelligent comments and moronism prevalent in this thread, I removed a line from this post to keep us on track of what this thread was meant to discuss.

There's another thing:
Recovery can't be used without /cache !
So we need to convert back each shutdown and convert to linux-swap each startup..

dumfuq said:
Couldn't we use mkyaffs2img (in recovery /sbin) to turn it back into yaffs?
Click to expand...
Click to collapse
I hadn't checked in the recovery img for any yaffs2 utilities. But if that indeed does exist then this will be possible.
So then the issue would be, is this safe to attempt long term with the amount of writes being done to internal flash?

maxisma said:
There's another thing:
Recovery can't be used without /cache !
So we need to convert back each shutdown and convert to linux-swap each startup..
Click to expand...
Click to collapse
So I see recovery is using it but the only thing in it is the recovery logfile. Could we modify the recovery to read/write the logfile to system/sd/recovery or something, or is the cache used for other stuff as well?

Another idea:
Instead of using it as swap, what if we could find a way to use it as compcache space? That way we don't sacrifice any of the internal ram for compcache and still have compcache running

maxisma said:
There's another thing:
Recovery can't be used without /cache !
So we need to convert back each shutdown and convert to linux-swap each startup..
Click to expand...
Click to collapse
What part of recovery needs /cache ? Does the recovery img mount the /cache partition? I'm wondering if the recovery img could be modified to write to the ext2 partition such as /system/sd/cache .

persiansown said:
Another idea:
Instead of using it as swap, what if we could find a way to use it as compcache space? That way we don't sacrifice any of the internal ram for compcache and still have compcache running
Click to expand...
Click to collapse
This is actually a good idea and it'd be interesting to see which way would benefit us more.

shafty023 said:
What part of recovery needs /cache ? Does the recovery img mount the /cache partition? I'm wondering if the recovery img could be modified to write to the ext2 partition such as /system/sd/cache .
Click to expand...
Click to collapse
When you activate an option in recovery, for example install update.zip, it writes the command into /cache/recovery and does a "quick reboot", which reads out the command and does this.

ok first dont even try it would one
reduce life on nand
cause issues with flashing
slow down any thing accessing the nand.
to the person thinking the ram is using flash. you must have never been to high school cause even typing class explains what ram is. plus g1 has memory write speeds of about 120mb/sec thats sdram. swap will never replace ram 120mb/sec versus 6 is huge

if you want real improvements you must change the code to something more efficient.
less bloat
more native code
fewer clock cycles
improved app states(cache on sd that would save the state by dumping part of ram to load later reducing clock cycles and reducing ram)
and my favorite rewrite system libs and apps to include more native code and reducing ram usage.
btw native code is about 10-100x faster than android java. that means 10%-100% more speed(or less load)

you could make a swap file and put it on there and just mount the file...........

yeah what he said!
zenulator said:
you could make a swap file and put it on there and just mount the file...........
Click to expand...
Click to collapse
This sounds like a good idea, altho we would have to get compcache working with backing swap file.
do we know if the /cache speed is compaired to Class 6 microSD?

good luck hardheads

jokersax11 said:
ok first dont even try it would one
reduce life on nand
cause issues with flashing
slow down any thing accessing the nand.
to the person thinking the ram is using flash. you must have never been to high school cause even typing class explains what ram is. plus g1 has memory write speeds of about 120mb/sec thats sdram. swap will never replace ram 120mb/sec versus 6 is huge
Click to expand...
Click to collapse
Dude honestly get a life. I have a Bachelor's degree in Computer Science. And I could code in Intel Assembly accessing each and every register in memory while you're left googling what Intel Assembly code means. Maybe you should keep up with the latest technology. If you did you'd know there are all-in-one memory chips that support NOR, NAND flash, and RAM.
http://www.eetasia.com/ART_8800466798_499486_NT_ab9e537f.HTM

jokersax11 said:
if you want real improvements you must change the code to something more efficient.
less bloat
more native code
fewer clock cycles
improved app states(cache on sd that would save the state by dumping part of ram to load later reducing clock cycles and reducing ram)
and my favorite rewrite system libs and apps to include more native code and reducing ram usage.
btw native code is about 10-100x faster than android java. that means 10%-100% more speed(or less load)
Click to expand...
Click to collapse
Exactly what type of "native code" are you referring to? And do you have any idea what type of memory the G1 uses for RAM? Go back to your typing class.

Shafty did you see Zenulator's idea? That seems like it could work.. Does the phone ever format the cache partition?

[Q] What are the benefits of formating the sd to ext/swap etc?

I'm a Linux user so I know what swap, ext2, ext3 etc technically means. What I don't understand is what are the *actual* benefits of formating my SD Card to these formats. I've run into a lot of guides and how-tos but no one cares to explain why to do it.
So to sum up:
Does the Android System actually cares if I create these formats?
If it does, will I notice actual increase in performance?
What about apps2SD?
What kind of format do you recomend?
I have an HTC Desire and I'm using CyanogenMod, Oxygen and MIUI in rotation (means I replace one with another every couple months).
Thanks in advance!

aventinus said:
I'm a Linux user so I know what swap, ext2, ext3 etc technically means. What I don't understand is what are the *actual* benefits of formating my SD Card to these formats. I've run into a lot of guides and how-tos but no one cares to explain why to do it.
So to sum up:
Does the Android System actually cares if I create these formats?
If it does, will I notice actual increase in performance?
What about apps2SD?
What kind of format do you recomend?
I have an HTC Desire and I'm using CyanogenMod, Oxygen and MIUI in rotation (means I replace one with another every couple months).
Thanks in advance!
Click to expand...
Click to collapse
Faster SD Card speeds for app2sd, thats mainly it. If you don't use A2SD there isnt much point to it, but if you do it can be very useful. What some people do though are format their System and Data partitions to EXT2 or EXT4 so then the whole system is faster all around and is a large improvement over stock rfs.

fowenati said:
Faster SD Card speeds for app2sd, thats mainly it. If you don't use A2SD there isnt much point to it, but if you do it can be very useful. What some people do though are format their System and Data partitions to EXT2 or EXT4 so then the whole system is faster all around and is a large improvement over stock rfs.
Click to expand...
Click to collapse
Ok so if make an ext partition, the apps that are stored in this partition will run faster. That sounds fair. Should I have both ext and FAT partions? And if I do, how does Android know where to put the apps? Also how about swap?

aventinus said:
Ok so if make an ext partition, the apps that are stored in this partition will run faster. That sounds fair. Should I have both ext and FAT partions? And if I do, how does Android know where to put the apps? Also how about swap?
Click to expand...
Click to collapse
No need for swap, and yes EXT is automatically recognized by android for a2sd, but make sure your ROM supports it. You would have an EXT partition between 256mb-2gb max, and then the rest would be FAT.

fowenati said:
No need for swap, and yes EXT is automatically recognized by android for a2sd, but make sure your ROM supports it. You would have an EXT partition between 256mb-2gb max, and then the rest would be FAT.
Click to expand...
Click to collapse
Ok! That was really eye-opening! Thank you so much!

no problem!

G2x sd configuration thread

Yes, I have searched; don't give me the whole guilt crap
I would like to know a couple of things...
1. Internal phone storage & phone memory, what is the difference and is there any mods people are doing with them as far as combining or something?
2. SD partitioning... are people still doing this and if so what are you recommending?
3. Swap... probably a thing of the past, but just would like to know.
Class 4 32gb
Running Weapon G2X
Faux stock kernels
Clockwork EXT4

alright lets make this world a better place, i'll tell you what i know
internal storage, is basically yours to use, add your music pics or whatever, just like how u use an sd card, phone memory aka application memory is reserved for apps only, no way of combining them, dont need to anyways, sd partitioning i never tried but i think its says it all, dividing your sd card into two or more partitions..make seem like you have two or more, u can reserve one just for music or pics or roms
and what do u mean by swap??
ooh yah if i helped a little hit the thxs button, imma collect now too lol

I came from from the G1 where it was standard practice to partition into fat32/ext/swap. Since moving to the G2x I have not needed it. Phone is super fast without swap and more than enough space for apps.
As for combining internal and external SD cards, I don't think I've seen that yet...
Sent from my LG-P999 using Tapatalk

I've never seen them combined but don't know if you can because if you remove the microsd, it would get all screwy.
I don't even use internal unless the rom does. I have a collection of apks/walls/music and my titanium files on my SD... everything else is expendable.
Sent from my LG-P999 using xda premium

player911 said:
I've never seen them combined but don't know if you can because if you remove the microsd, it would get all screwy.
I don't even use internal unless the rom does. I have a collection of apks/walls/music and my titanium files on my SD... everything else is expendable.
Sent from my LG-P999 using xda premium
Click to expand...
Click to collapse
and with those on the SD are you using them just anywhere on the SD or do you have an EXT 4 partition?

dylyxcore said:
Yes, I have searched; don't give me the whole guilt crap
I would like to know a couple of things...
1. Internal phone storage & phone memory, what is the difference and is there any mods people are doing with them as far as combining or something?
2. SD partitioning... are people still doing this and if so what are you recommending?
3. Swap... probably a thing of the past, but just would like to know.
Class 4 32gb
Running Weapon G2X
Faux stock kernels
Clockwork EXT4
Click to expand...
Click to collapse
Internal phone storage is where your system files are stored. It is also where apps from the market are installed. Phone memory is utilized by the stock roms as the default "sdcard". By default this is where your photos, music, documents, etc are stored. There is also an external slot for a physical sdcard. This is actually mounted to /sdcard/_ExternalSD with a stock rom.
There is no need to partition the external sdcard. There is enough space on the phone by default and it probably wouldn't be worth the hassle to make it work.
I don't think you need swap with this phone.

phburks said:
Internal phone storage is where your system files are stored. It is also where apps from the market are installed. Phone memory is utilized by the stock roms as the default "sdcard". By default this is where your photos, music, documents, etc are stored. There is also an external slot for a physical sdcard. This is actually mounted to /sdcard/_ExternalSD with a stock rom.
There is no need to partition the external sdcard. There is enough space on the phone by default and it probably wouldn't be worth the hassle to make it work.
I don't think you need swap with this phone.
Click to expand...
Click to collapse
Thank you for clearing this up, so in clockwork recovery why does it support EXT 4? Is that just the format it saves a back-up in?

dylyxcore said:
Thank you for clearing this up, so in clockwork recovery why does it support EXT 4? Is that just the format it saves a back-up in?
Click to expand...
Click to collapse
No. Ext4 is a type of filesystem used commonly in Linux, just like FAT and NTFS are used with Windows setups. There are some roms that will mount your system as, or even fully format your system into a ext4 filesystem. Clockwork supports that filesystem, meaning it can be used to install ext4 roms.

phburks said:
No. Ext4 is a type of filesystem used commonly in Linux, just like FAT and NTFS are used with Windows setups. There are some roms that will mount your system as, or even fully format your system into a ext4 filesystem. Clockwork supports that filesystem, meaning it can be used to install ext4 roms.
Click to expand...
Click to collapse
Thank you for your help, one last thing... to run these do you need an EXT 4 partition, or is that completely different? What makes the ROM to be EXT 4?

dylyxcore said:
Thank you for your help, one last thing... to run these do you need an EXT 4 partition, or is that completely different? What makes the ROM to be EXT 4?
Click to expand...
Click to collapse
No you simply flash like you would any other rom. If the rom actually formats your drives to ext4 i beleive you will have convert back to ext3 in order to flash an ext3 rom. There is a conversion utility somewhere on here if you search around. Someone else wanna chime in to verify about the need to convert back to ext3?

phburks said:
No you simply flash like you would any other rom. If the rom actually formats your drives to ext4 i beleive you will have convert back to ext3 in order to flash an ext3 rom. There is a conversion utility somewhere on here if you search around. Someone else wanna chime in to verify about the need to convert back to ext3?
Click to expand...
Click to collapse
Yeah I read about the EXT 3 to EXT 4 dilemma on here via CM7 when it was upgraded from an overnight build or something... I think it had to do with restoring apps or data? Really appreciate your information, this finally clears up the confusion
p.s. thanks'ed all your replies

Regarding disk space

Has anyone tried to repartition our /system and /data folders to create an /emmc partition for our phones? Seems like OE partitioning scheme is a huge waste of space and we could better utilize the extra space if it was in user mountable partition.
This should be easy to do via recovery yes?
Sent from my MB865 using xda premium

unsivil_audio said:
Has anyone tried to repartition our /system and /data folders to create an /emmc partition for our phones? Seems like OE partitioning scheme is a huge waste of space and we could better utilize the extra space if it was in user mountable partition.
This should be easy to do via recovery yes?
Sent from my MB865 using xda premium
Click to expand...
Click to collapse
Yes both lfaber06 and I sort of do something similar, with the way we have safestrap setup, but we use the SDcard instead. The one issue I see with this is what the future brings for this phone, like what moto might do for us with ICS, and such, what may be needed in these partitions to make it work. Other that that I don't see an issue, but you will have to change a ton of start scripts if the partition numbers change.
Jim
Sent from my MB865 using xda premium

I don't think you understand, I want the space to be user accessible, like I have on my nook. I flashed a custom repartition from cwm to resize data and media partitions. The /emmc partition is a fat32 partition I believe (on my nook anyways). I've never even come close to filling my 2gb data partition, and now I have 4gb of space for music or videos storage.

unsivil_audio said:
I don't think you understand, I want the space to be user accessible, like I have on my nook. I flashed a custom repartition from cwm to resize data and media partitions. The /emmc partition is a fat32 partition I believe (on my nook anyways). I've never even come close to filling my 2gb data partition, and now I have 4gb of space for music or videos storage.
Click to expand...
Click to collapse
No, I don't think you understand what that really does under the covers. Both lfaber06 and I have been working on this device for a while, we are the ONLY two devs currently working on this device. What I am telling you is that if you change the size of ANY of the internal partitions, It might effect the ability to update to another version of Android. Your Nook, is not a Mototrola device, and Moto has completly messed with/up android, and therefore it is not safe to do what you are saying. No matter what you can not get access to the internal partitions, yes on AOSP devices / devices that can be AOSP (the nook is one), you can do those things. The Atrix2 still has a locked bootloader, so we are stuck with their kernel, and version of android. To do what you are saying you will need to have an AOSP kernel, and have MANY init scripts updated to understand the new partition table. It is not as easy as you make it out to be.

Okay, I was under the impression with rw abilities on /system and /data partitions we also have the ability to mount, unmount; also giving us the ability to edit the partition blocks via bootstrap.
Ok how about this, there is a guy in the Nook Tablet forum (locked bootloader, like us) who had the idea of basically running a fat32 "img" that in the /data (ext4) partition to that would mount via a script and be user accessible? That would be possible I would think.

unsivil_audio said:
Okay, I was under the impression with rw abilities on /system and /data partitions we also have the ability to mount, unmount; also giving us the ability to edit the partition blocks via bootstrap.
Ok how about this, there is a guy in the Nook Tablet forum (locked bootloader, like us) who had the idea of basically running a fat32 "img" that in the /data (ext4) partition to that would mount via a script and be user accessible? That would be possible I would think.
Click to expand...
Click to collapse
Yes you do have access to all of those things with root, but Moto has done so much customization to this thing, that would not be safe.
Yes, that is also something that lfaber06 and I have thrown around as well, the logistics of it need to be worked out for our phone, and the proper init scripts need to be put together.
Are you up for the task? I only ask because he and I are currently working on porting cm7 and cm9 to our phone, which is a huge task. I can help you out, but I just can't take on anymore projects for this phone at the moment, since I have 5 different things I am working on for us right now.

I am definitely down to help, still going to be a bit of a learning curve for myself. Still pretty new to linux/android.

unsivil_audio said:
I am definitely down to help, still going to be a bit of a learning curve for myself. Still pretty new to linux/android.
Click to expand...
Click to collapse
PM me, with as much info as you have about this, preferably any links that you have read, and I will help you develop a game plan, and what you will need to make this hack happen. You will have to do some research on your own, and I can help with the Linux/Android part, and help you figure out what you will need to do. I am also not sure on the what kind of speed to expect from something like this, so that will also be something to look into, since this will be a virtual disk file.

[Q&A] 2 GB External Data -- Hefe Hook Kernel and others

The Hefe Hook kernel allows you to mount a partition of your microSD as /data, getting 2 GB (or more) for your apps and their data.
Please ask your questions here about installation, use, or general approach.

This is great @jeffsf and can u show me how to re-partition the "real internal" storage? I mean expanding the /system since u put the /data out of it. Thanks man
Sent from my GT-N7000 using Tapatalk 2

daothanhduy1996 said:
This is great @jeffsf can u show me how to re-partition the "real internal" storage? I mean expanding the /system since u put the /data out of it.
Click to expand...
Click to collapse
Yes, you've hit on another of the reasons I've been exploring using the microSD for "live" storage. It's one thing to use potentially slow storage for photos that you basically write once or "app to SD" where you read the APK at boot time, make sure your ODEX in the Davlik cache is good, then generally read from the internal-flash cache. It is another when that memory is being read and written "constantly" when your phone is running.
The good thing is that once /data is not part of the internal flash filesystem, you don't have to worry about one ROM (kernel) thinking it begins at one place and another saying it starts at another. Previously, if one ROM had one /system size and another and a different one, the next partition, /data, would look to be corrupt when you swapped ROMs.
As a warning, not all users have fast microSD cards. Some that say "Class 10" on them really are dogs, especially for small reads/writes. The "Class" ratings are for sustained writes, as you would have with a camera recording video. If your ROM is "external /data" only, or even defaults to that, be prepared for a slew of "Your ROMs sucks. It is so slow." complaints.
You'll also need a way to automate formatting the card. It can be done on the phone, as long as you aren't trying to preserve any data.
As I recall, the layout of the MTD partitions is done in drivers/mtd/onenand/samsung_galaxys4g.h I would be careful not to move the partition boundary for efs, as you'd have to move the data it contains in your updater script. Repeating the warning about not moving the boot and recovery partitions is probably a good idea as well!
Your build tree may need some of these values, or at least think it needs some of these values. For example, device/samsung/aries-common from the CyanogenMod/cm-11.0 (KitKat) branch calls out NAND page sizes, partition sizes, and flash block sizes. I haven't looked in detail at your build tree so I can't comment on how it might handle things differently than the CyanogenMod one.

itzik2sh said:
Hi
I hope I don't ask anything silly, but please let me know if any of my assumptions is wrong :
1. I take FBi's251's AOKP milestone 6 (ICS 4.0.4)
2. 8GB SDCard was formatted to FAT32 (4GB) and EXT3 (4GB) using TWRP kernel
(Beastmode's proton kernel to be exact).
3. I would flash this kernel and it would move apps and their data to the sd-ext
without any special additions.
Thanks. I read the thread, but unfortunately 8GB SD is what i have and I think it should be enough.
Thanks again.
P.S - it's for 2 guys I already sold them my SGS4Gs. I want them to be happy...
Click to expand...
Click to collapse
AOKP should be fine. I haven't tried it recently, but it was the tree in which I did the early Hefe Kernel development.
As I understand it, you have a microSD with
partition 1: 4 GB FAT32
partition 2: 4 GB ext3
So that can work, but will need some tweaking of the init-on-fs.sh script. I intentionally didn't use the second partition as so many scripts gobble that up as sd-ext and do who knows what to it.
My first preference would be to reformat the cards, perhaps:
6 GB FAT32
1 MB ext2/3/4 (Yes, 1 MB, a sliver, choice of ext2, ext3, ext4 up to you)
2 GB ext4
since then the script will work without modification and if they install a third-party script that uses the second partition, it won't corrupt their data.
If you were to keep the formatting the way that it is now, you'd need to edit the mount commands in the script to look something like:
Code:
/system/xbin/busybox umount /data
/system/xbin/busybox mount -t ext3 -o noatime /dev/block/mmcblk0p2 /data
(removing the sd-ext mount)
I'm not sure what your expectations are, but all that the kernel and that script will do is mount a different disk partition on /data -- you need to manually move the data over (or restore from something like Titanium Backup). There may be some trickery in renaming that could be used with TWRP backups to restore from data.yaffs2.win to the new /data partition, but I haven't tried that at all.

Hi Jeff
Thanks for your quick reply, and sorry again for not seeing the Q&A thread.
I think making it :
partition 1: 4 GB FAT32 (sdcard)
partition 2: 2 GB ext3 (sd-ext)
partition 3: 2 GB ext3 (data)
partition 4: 1 MB (spare)
would be better and handle data as well. don't you think ?
Would it be worth doing with a "Team" micro-SD card (class 6 I believe) ?
Thanks.

I haven't tried a Class 6 card, but my gut feeling is that it will be dicey. I didn't "commit" to using /data on microSD until I had tried it for several days using Titanium Backup's ability to move both apps and app data to the external card. I would try that first, especially as the phones in question aren't going to be in your hands (I consider you an expert user, able to manage things outside the UI with ease).
I've attached some testing I did a while ago with Transcend and SanDisk cards. When you look at them, realize that the speed scale changes between them. I have a feeling that the real "performance" on a device is going to be related to relatively small reads and writes, not the ability to stream video to the card. I also don't know much about the Team brand, but I found that even some well-known brands didn't have the performance of the Transcend or SanDisk in the same category.
However you configure your cards, I would definitely recommend a journaling filesystem of some sort. I've had my microSD come loose inside the phone. The journal will at least help to reduce any filesystem corruption should that happen.
You don't need the fourth partition -- I have it there to be able to keep rsync backups for fast ROM swapping.
.

Regarding the apps data, have you tried exploring the Mount2SD script ?
sent from me

I've tried a couple of the scripts out there in the past. Since backing up my data is very important to me, I trust the scripts in Titanium Backup to work well with its backup/restore strategies.
Mounts2SD looks like it has gotten a lot more sophisticated than it was when I tried it in the past. It sounds like something worth trying in its current state. At a quick glance (and not looking at the code), I'd personally make some different choices about features; enabling journaling, and being concerned about why lost+found was filling up (things should only appear there if the file system is found to be corrupt).