[DATA2EXT] [11/Dec] FULL BENCHMARK, V4, Improve the efficiency of IO - Desire Android Development

This script only for ROM Developers!
EDIT:2010/12/13 update explain:
1. Why three partition?
Answer:for get best balance of stabilty and performance.
2. Why don't use two partition and loopfile on 2nd partition(my script V1)?
Answer:Loopfile on 2nd partition need enable journaling on 2nd partittion file system to get stability after Unexpected loss of power or Unsafe reboot(without umount). And, enable journaling on a large partition is slower than small partition. In my phone, 1.3G /data/app don't need journal, only 300M /data/data need journal, so I use a small parition for safe and performance in V4.
3. Why not use ext4 directly at 3rd partition?
Answer:for speed up sqlite3, ext4 loopfile on ext4 device, make a huge improve for sqlite3 insert.
4.Why not NILFS2 or BRTFS?
Answer: I have already tried nilfs2 before I use ext4. nilfs2 crash my system. Brtfs marked 'EXPERIMENTAL' in kernel source(2.6.32.15), I like stability more than speed.
________________________________________________________________________________________
EDIT:2010/12/12 update V4 script:
/data/:ext4, disable journaling
/data/data:ext4 loopfile on ext4 device, enable journaling on ext4 device, disable journaling on ext4 loopfile, for fast and stable sqlite3 access.
really smooth this time. please trying...
________________________________________________________________________________________
EDIT:2010/12/11 FULL BENCHMARK: (all test use my sandisk C2 16G TF card, run in SuSE Linux 11.2 desktop)
I did a full benchmark for ext2/ext4, with loop or without loop, write, read, sqlite insert, this is result:
I do follow tests, umount after every test to prevent from kernel cache.
Sqlite3 db benchmark is insert 2000 records to db.
500 50k-2M files(random size) to 2G ext4 loopfile on ext4 device, enable journaling on ext4, sync after every file write.
write: 355 seconds
read: 52 seconds
db:76 seconds
500 50k-2M files(random size) to 2G ext4 loopfile on ext4 device, disable journaling on ext4, sync after every file write.
write: 212 seconds
read: 52 seconds
db:3 seconds
500 50k-2M files(random size) to 2G ext2 loopfile on ext4 device, with sync-in-mount-option, sync after every file write.
write: 380 seconds
read: 45 seconds
db: 3 seconds
500 50k-2M files(random size) to 2G ext2 loopfile on ext4 device, without sync-in-mount-option, sync after every file write.
write: 390 seconds
read: 45 seconds
db: 3 seconds
500 50k-2M files(random size) to ext4 device, enable journaling, sync after every file write.
write: 208 seconds
read: 47 seconds
db: 95 seconds
500 50k-2M files(random size) to ext4 device, disable journaling, sync after every file write.
write: 191 seconds
read: 48 seconds
db: 50 seconds
500 50k-2M files(random size) to ext4 device mount with loop option, enable journaling, sync after every file write.
write: 198 seconds
read: 51 seconds
db: 72 seconds
500 50k-2M files(random size) to ext4 device mount with loop option, disable journaling, sync after every file write.
write: 191 seconds
read: 50 seconds
db: 2 seconds
As you see, the best speed is: mount real ext4 device with loop option, disable journaling. The best balance of performance and safe is mount real ext4 device with loop option, enable journaling! We don't need a loopfile but just change mount option.
I attach my testsuit file below.
___________________________________________________________________
EDIT:2010/12/10 V3 changelog:
1.change loopback device to real partition for powersave and better write performance.
2.disable second partition's journaling.
3.You need 3rd partition for /data/data. But even without the third partition of this script can also work(not stable). My partition settings:second partition:1.5G, thrid partition:0.5G
Explain:
1.about NAND and SD: Desire have poor NAND memory for userdata, just 150M(this can be enlarged by MTD or HBOOT, but still small for me). This MTD partition need store app, dalvik-cache, app-data, it is too small to fill many apps.So someone think may be someting can be symbol link to SD, this is original Cyanogen A2SD, in this case , the /data/data directory is not moved to SD, because the battery reason and the speed need. This is what the HDROM 2NAND version do. So, the original A2SD is totally same as 2NAND. I don't known why the 2NAND be named. The NAND is not fast because it is yaffs2(journaling) filesystem, so if install many apps the phone will be slow(every app's sqlite have write demand)
2.about ext2/ext3/ext4: Ext2 have no journal feature, so lost power or just normal reboot will be crashed it, ext3 is journaling version of ext2. Ext4 is diffrent from ext2/ext3, it included Extends feautre, which provide better performance for large file(large than block size:4K). I analyze android system files, 4/5 is large than 4k, so the Extends feature is very useful. all EXTX system is base on Block, and the default block size is 1024 for small partition. Larger block size has better performance, so I force block size to 4096 when format.
3.about loopback device: The original Data2SD script make a huge loopback file for /data parition, then get better IO score. The reason is loopback device provide better Read performance. But the Write performance is wrose, so the phone got lag when install app or copy file.And the lag will occur suddenly. If system want to write data to loopback device, it is not real write, just write to kernel buffer/cache, the data will be flush to real disk when real filesystem(where loopback device stay on) got sync or commit command. So the real filesystem must enable journaling or do sync frequently。
4.about my old data2ext script: I try to splite directory to diffrent FS for the balance of performance and lag. Why I formated partition to EXT4? the reason can be found at item 2. Why create a small loopback device? the reason is item 3. But I found some bug with this case, any data write to loopback device need twice kernel IO command, one virtual one real, so the cpu usage will be high. Real EXT4 partition don't need journaling if no loopback device on it. For data integrity on virtual EXT4 loopback device, I had to enable journal on it.So I design new disk layout for better performance, it is DATA2EXT V3.
__________________________________________________________________
V2 change log:
1.change loopback device FS to EXT4 for journal, more stable
2.change mount option, add commit=large Prime number, reduce real IO, avoid concentration of commit. change journaling scheduler from Ordered to WriteBack to improve performance.
I would like to share my Data2EXT script, a collection of data2sd(Thanks to sibere) and the advantages of ext4, you can let the phone run at peak efficiency.
I use EXT4 for whole SD 2nd partition, and create a ext2 loopback device with 1/5 size of 2nd partition.
The files in /data/app,/data/.systemapp it not updated frequently. so store them in EXT4.
In /data/data and /data/app-private, a lot of small files and sqlite database, is updated frequently. so store thenm in EXT2 loopback device.
/data/dalvik-cache is stored in phone memory /mnt/asec/mtddata for distributed IO pressure.
/data/misc, /data/system, /data/property is stored in phone memory /mnt/asec/mtddata for system settings restore(or you will lost your timezone/language settings after reboot).
use this script, I can get quandrant score:2000+, IO:4000+, my phone is smooth and quickly.

this is great job
thx ownhere

If I have Data2SD already installed, I have to do it again, or simply copy this file to /system/etc/init.d and reboot?

I do not understand that I do, only copy the script and reboot and automatically makes the whole process or do I have any other operation, thanks and apologies.
PD. I have read and believed the other post, which is to automatically process, the only problem was the FC, which was corrected by manager rom also lose the wifi, this script solves these problems?

Do you need to use any and on this and can u use an ext2 partition
Sent from my HTC Desire using XDA App

noob question.. what do i do with the script?

http://forum.xda-developers.com/showpost.php?p=9541647&postcount=477

ownhere said:
I use EXT4 for whole SD 2nd partition, and create a ext2 loopback device with 1/5 size of 2nd partition.
Click to expand...
Click to collapse
I don't understand this......Can you explain it?
How partition the micro_SD card?

Can someone explain how to apply the script? Do we have to create the partitions ourself or will the script create them?
Does it work on DHD Roms?

Guys there's a thread only god knows how many pages long that explains every possible variation on EXT* loopback devices..how about scrolling just a little next time?
http://forum.xda-developers.com/showthread.php?t=852219

I think that is better always insert an how to guide!
As it is, is not very clear :-(
sent from the future

Good job ownhere, I want to try your script on AuraxTSense, but this rom don't have mkfs.ext4 or mkfs.ext3, can you attached it to 1st post please?

+1 for a bit more detailed information.
I am not requesting, I am just politely asking with gratitude already since I guess this script will be implemented in future roms to come i guess..

shawe_ewahs said:
Good job ownhere, I want to try your script on AuraxTSense, but this rom don't have mkfs.ext4 or mkfs.ext3, can you attached it to 1st post please?
Click to expand...
Click to collapse
Try /system/xbin/busybox mkfs.ext4

coutts99 said:
Try /system/xbin/busybox mkfs.ext4
Click to expand...
Click to collapse
I try this before ask, with version 1.17.1 don't work for me, requires 1.18.0 for work?
# /system/xbin/busybox mkfs.ext4
mkfs.ext4: applet not found

shawe_ewahs said:
I try this before ask, with version 1.17.1 don't work for me, requires 1.18.0 for work?
# /system/xbin/busybox mkfs.ext4
mkfs.ext4: applet not found
Click to expand...
Click to collapse
uploaded in 1st post.

ownhere said:
uploaded in 1st post.
Click to expand...
Click to collapse
Thanks ownhere!

busybox ---> /system/xbin
mkfs.ext4 ---> /system/xbin
01data2sd ---> init.d
yes i try again report back

It works well.
35.121 on LINPACK @ 998Mhz

bbk said:
It works well.
35.121 on LINPACK @ 998Mhz
Click to expand...
Click to collapse
how do you make this work?
i am now at splash for about 10min

Related

[Q] ext2/ext3/ex4 and lynux-swap with incubus RLS14/15

Hi, I have been testing several NAND set-ups
-system to NAND and data to resized data,img
-system to resized system.img and data to resized data.img
-system to NAND and data to ext2
...
I am currently using NAND and data to ext2 partition, so all apps are installed there. After lots of reading I can not understand posts that are obsolete and posts that are not so I would appreciate if someone can explain me the basics and optimal setup for incubus superfroyo RLS14/15 roms:
I have now 4 partitions:
1) fat32,
2) 1,5G ext2 (app, app-private etc),
3) 1,5G ext2 (lost+found folder) and
4) 96M lynux-swap.
- How can I turn ext2 to ext3? do I need to do so? I have also read about ext4. Upgrading to ext3/ext4 applies to my 2) partition, to my 3) partition or to both? will I loose data stored? "there is no ext3/ext4 fs because there isn't the support on the compiled kernel. Ext3/ex4 are journal file system so are bad for limited writing device like nand or sd and thre is no good rease to use it" answered by l1qd1.
- How can I check if my lynux-swap is enabled and used by the rom?
- Do partitions 2) and 3) really need to be <1,5 G? why so?
- Can I take apk from apps folder in partition 2) in case I want to keep them for future fresh installs?
- What is lynux-swap optimal size? a 1,5G partition would be worse than 96 M? Why?
- Which tools are you using with incubus roms to manage installaion location and similar functionalities?
Another big mark I have is about deoxeding installed aps (not belonging to the ROM). Should I do it? why? how?
Pretty basic stuff I guess for experienced users & programmers, but I am new to lynux and android. Have read lots of things but it is difficult to learn by myself what would work with our Polaris and incubus ROMS and dzo/kk kernels,
Stick to ext2 man, you won't turn your old engine to 240mph on a Ford Model T.
You're not getting some major boost if the hardware dosn't use it, so I'd stick with ext2.
And you also want Super FroYo on your ext2 partition, yes? I'm here to help.
- How can I check if my lynux-swap is enabled and used by the rom?
adb shell
Code:
free
if on swap you have 0 it isn't enabled.
Code:
swapon /dev/device
to enable it or i suggest you tu use compcache.
- Do partitions 2) and 3) really need to be <1,5 G? No
- Can I take apk from apps folder in partition 2) in case I want to keep them for future fresh installs? Yes but not all apk support this
- What is lynux-swap optimal size? a 1,5G partition would be worse than 96 M? Why? Too Big = more sparse but it's a problem only for real disk so you can create a swap big as you want but it's unuseful (100 M is enaught)
i suggest you to switch to my kernels because are more updated and kk move to another device.
Yah l1q1d I love ur kernels. Eager for more !
l1q1d, I am testing your kernel as you advised me. I could apply the wifi patch without problems using your androiupdate18-09-10, as you told me some time ago they are both 2.6.25.
However I do not manage to turn linux-swap on: that's what I do: I install ubuntu, I use G-Parted to create partitions on the sd card. When I do so, the third partition is always shown as linux-swap in G-Parted but when I test adb shell -> parted -> print, it always shows as ext2. I can not dismount and can not swap on it.
The other way round, if I deleted all partitions with G-Parted and create the partitions with mkpartsfs or if I use mkfs to change the third partition from ext2 to linux-swap, then sometimes I can swap on it (directly on adb shell or with G-Parted) but when I unplug the device will tell me the card is empty and will not be able to mount it, access it from ubuntu or from the device. If I reboot then, the system will hang until I install both data and sysetm to nand. This happens no matter if I activate or not from the polaris or even if I switch on or off usb mode. The card at that point is useless. I guess the partition table or some other basic file is messed.
Could you please explain a little more how can I come with a 13.5G fat32 partition, a 2,4 ext2 partiions and a 0,1 lynux swap swapped on partition??
I really need to test this set up, since with the standard set-up (system to nand, fat32 for files and ext2 for installed apps I can not install all apps I use since performance drops significantly).
regards
well, ... dumb me, maybe with compcache you are referring to launch a script?
where do I get this compcache.sh? how do I launch it? from adb? from the terminal? from the sd card?...

[Utility] ext4 data2sd for Froyo ROMs

If you are always low on program memory, this data2sd might be for you. It works with any Froyo for the G3.
Note: This won't wipe your data, but you should have a backup, just in case. First backup the phone to the SD card, then the SD card to your computer. Be cautious. I give no guarantee whatsoever. If your phone behaves funny, even eats your cat afterwards, I take no responsibility.
Note: starting with version 3, the new storage size will NOT be reflected in the Task Manager or in Titanium Backup. I'll see what I can do about it.
You need a rooted phone and an SD card, preferably 2 GB or more and class 6 or better (higher). data2sd doesn't make your phone faster. Much to the contrary: it extends your memory at the cost of speed. That said, if your SD card is of a class smaller than 6, this will work too. The slowdown, however, will be noticeable in some situations.
Three steps are needed. First, we need a kernel with built-in ext4 if you don't have one. Second, we bring the data2sd into its place. And third, we create an ext 4 partition with the desired size. Now follows a detailed description of these steps.
Step 1: Check your kernel
You need a kernel with built-in ext4 capability. The custom ROMs mostly have one. If you are running a stock kernel, you must install one, e.g. from http://forum.xda-developers.com/showthread.php?t=978823 . Follow the installation steps detailed there. (Changing the kernel does not wipe your phone).
Step 2: Install data2sd
You now must install data2sd onto your phone. As long as you didn't modify your SD card (as long as partition #2 is not ext4), this will have no further impact on your phone.
Download data2sd.zip from below and unzip it. There's a single file in there: data2sd.sh. Use the tool that suits you best to transfer it to the SD card of your phone. With adb, you would do
Code:
adb push data2sd.sh /mnt/sdcard
Now either start a terminal emulator with a root shell or use adb again. With adb, you would execute
Code:
adb shell
Make sure you are root (the prompt reads '#'), else run "su". Now type in:
Code:
# sh /mnt/sdcard/data2sd.sh
You will see some lines about what is happening and a final confirmation.
That's it. Again, your phone will continue to behave as usual.
Step 3: Prepare your SD card
You can use any amount of program memory up to the size of your SD card, but be sure to leave some room for application data on the normal (#1, vfat) partition. As a rule of thumb, 0.5 GB of program memory will satisfy most needs, 1 GB is plenty.
If you're running Linux, prepare a second partition (must be partition #2) with the desired size. Use the usual utilities to do so. This includes gparted, but there are also other means. If you use gparted, you can follow the description for Parted Magic below.
If you have no access to a Linux computer, even not your friend's, your (probably) best choice is to run Parted Magic from a live CD. For this, go to http://partedmagic.com, and burn your CD. Instructions on how to do it with different burning tools are also there.
Step 3.1: Partition and format your SD card with Parted Magic (also applies to gparted)
The following describes the steps to partition your SD card with Parted Magic. You don't need to use this method - use the tool that suits you best.
I assume you have gparted or Parted Magic running and your SD card inserted into the card reader of your PC. Now select your card from the drop-down menu of available drives in the upper right corner. You better double check that you're on the correct drive (card). It will happily erase your system disk if told to do so.
Right click near top into the block designating your current partition #1. Select "resize". (I assume that your partition #1 is the only one. If not, delete the other partition(s)). Change the size to the total size of your SD card minus the desired size for program storage (/data). Confirm your selection.
There is now a gray block designating free space to the right of your first partition. Right click it and select "New". In the following window, leave the sizes as they are. Leave "create as" to be a primary partition, and change "file system" to ext4. Confirm your selection ("Add").
The gray block at the right has changed to a colored frame. Right click it again and select "Format as". Select "ext4".
If everything went well, you can now click on the green check mark in the icon bar near the top, confirm, and watch Parted Magic resizing your normal (#1, VFAT) partition and creating the new ext4 one.
4. Finale furioso
Now insert the SD card and reboot your phone. The Task Manager will report the new size in its rightmost tab, as do the other utilities like Titanium. All user apps and data should be in place. All will look like you have soldered in an extra chunk of internal memory. Despite not being directly visible, your program memory is resized to be the new partition. You see its size and its usage (in percent) by running
Code:
df
from adb or from a terminal emulator. Have a look at the subdirectories of /data displayed there.
As a final step, you can safely remove data2sd.sh from your SD card (/sdcard). It is not needed any more.
Q&A
* How can I remove my SD card while the phone is on?
You can't. Don't try. Always switch your phone off before you remove the SD card.​* How can I return to the state before data2sd?
Remove the SD card. All user data and apps will be there in the state you left them before you installed data2sd. The data2sd script you have installed simply does nothing as long as there is no ext4 #2 SD partition.
If you want to reuse the SD card, remove partition #2. This can be done with each partitioning tool, you don't need Linux.​• What about app2sd from Kyrillos 3.0? Is it compatible?
Yes. But "no" for the "no wipe" version which was offered for a short period of time. In the latter case, clear out the ext4 partition (important: do not forget the hidden .data2sd_initialized file) or simply reformat it before reusing it.​* It doesn't work. What can I do to help diagnose the problem?
See if the file /mnt/asec/data2sd/log.txt exists. If it does, have a look at it and try to make sense of it. If it doesn't tell you anything useful, add it to your error description in the forum here. If there is none, mention that in the description.​* What things must be considered if I flash a new ROM?
As always with such actions, make sure you have a backup. That said, just flash the new ROM and then install data2sd. When you reboot with the SD card in place, all user Apps and all data should be there. No restore from backup required.
Only in the rare case that the new ROM explicitly requires you to NOT restore your old data, e.g., from Titanium, you should reformat your ext4 partition (to clear it out) and proceed as advised by the ROM makers.​Versions:
2011-03-19 Initial release
2011-03-21 Version 2
- Updated for ROMs without /system/etc/init.d
- Simplified installation of the core parts
2011-03-23 Version 3
- Vibration bug fixed
- App2SD mode added: If the ext4 partition bears the volume label "app2sd" on a new blank SD, only /data/app is moved. This greatly reduces size benefits, but results in nearly no speed loss.
- Just run the data2sd.sh file (as described in step 2) to upgrade without data loss.
- Note: the actual size will NOT be reflected as described in the Task Manager or in Titanium.
Gosh this sounds so complex and risky! Isn't there a easier way?
☞★sent from my Galaxy3!!
♪drajax!! ♬♂ツ
drajax said:
Gosh this sounds so complex and risky! Isn't there a easier way?
☞★sent from my Galaxy3!!
♪drajax!! ♬♂ツ
Click to expand...
Click to collapse
Flash Kyrillos' v3.1 where's the script's allready built in...
It is less complex than it sounds. The description is just really very detailed. The details may sound confusing until you try it. And if you have access to Linux, it is really simple. We Linux users also have to use a different operating system when we flash our phones.
Alternative: Install Kyrillos 3.1. It contains the same data2sd and comes with a partitioning tool. But, as far as I can tell, you need to reformat the SD card's vfat part on the phone and lose the data there (not sure, however).
when i try to set permissions it says permission denied . Any help?
http://www.appbrain.com/app/move2sd-enabler-(root-only)/com.leinardi.setinstalllocation
Just one quick question....
I am having the stock JPM ROM... in that case do i need to have some new Kernel to use your data2sd???
I'll be using this!
Sent from my ACID Lestatious v1.4.5.5 Galaxy 3 FROYO
When i write down "/ # cd /system/etc/init.d/"
it says "sh: cd: can't cd /system/etc/init.d/"
What am i doing wrong???
Pinak.ahuja said:
when i try to set permissions it says permission denied . Any help?
Click to expand...
Click to collapse
You're not in a root shell. Enter "su" as your first command.
next2devil said:
When i write down "/ # cd /system/etc/init.d/"
it says "sh: cd: can't cd /system/etc/init.d/"
What am i doing wrong???
Click to expand...
Click to collapse
Don't know. Try this:
cd /system
cd etc
cd init.d
and report where it fails and if you are using adb or a terminal emulator. If it doesn't fail, continue with the line after the cd command.
Awesome work mizch!
We can refer to below link for partitioning of sd card to ext4 using adb
http://forum.xda-developers.com/showthread.php?t=534714
Hi Mizch,
Thanks for the script. have few questions; requesting for an answer:
1. Once data2sd is done, will sd card be auto mounted on reboot?
2. I am running JPM with Apollo's data2sd (ext2 partition for apps 1.1GB). What will be the benefit of ext4 over ext2?
3. I understand that i have to remove apollo's lagfix and then apply ext4 data2sd. Is there a way, i can also apply lagfix?
Thanks
jaskiratsingh said:
Hi Mizch,
Thanks for the script. have few questions; requesting for an answer:
1. Once data2sd is done, will sd card be auto mounted on reboot?
Click to expand...
Click to collapse
Yes. And for the first boot, it gets initialized too.
2. I am running JPM with Apollo's data2sd (ext2 partition for apps 1.1GB). What will be the benefit of ext4 over ext2?
Click to expand...
Click to collapse
Reliability, especially in crash cases and generally when the phone is shut down without a clean unmount. Ext 4, being a journaling file system, tries to keep the file system in a consistent state whenever possible. ext2 does nothing like that. So, when you forcible switch off your phone by long pressing the power button, most likely you are continuing with a damaged file system whose errors accumulate from now on and will eventually become visible at some time later.
There's one drawback, however: ext4 has to write more data than ext2. ext4 must always write the information needed to reconstruct a consistent file system to its journal.
3. I understand that i have to remove apollo's lagfix and then apply ext4 data2sd. Is there a way, i can also apply lagfix?
Click to expand...
Click to collapse
If I'm correct, the lagfix converts /data (your program storage) to ext2. Since /data is now handled by data2sd and converted to eyt4 anyway, you cannot use lagfix any more. You need to remove it.
mizch said:
If I'm correct, the lagfix converts /data (your program storage) to ext2. Since /data is now handled by data2sd and converted to eyt4 anyway, you cannot use lagfix any more. You need to remove it.
Click to expand...
Click to collapse
Thanks for the quick reply mizch. If i have to check apart from data, if anything else is converted into ext2, how can i do that?
jaskiratsingh said:
Thanks for the quick reply mizch. If i have to check apart from data, if anything else is converted into ext2, how can i do that?
Click to expand...
Click to collapse
have a look at the output of 'mount' or 'busybox mount'.
rm 02-app2sd for who use kyrillos ROM...i didn't do that command..but it seems works fine...what does it change if i use that command???
mizch said:
Don't know. Try this:
cd /system
cd etc
cd init.d
and report where it fails and if you are using adb or a terminal emulator. If it doesn't fail, continue with the line after the cd command.
Click to expand...
Click to collapse
I tried what you mentioned.... but again it shows the same error. It works well till "cd etc". Then, for init.d it shows out the error.
It seems the location for init.d is different in my device
next2devil said:
I tried what you mentioned.... but again it shows the same error. It works well till "cd etc". Then, for init.d it shows out the error.
It seems the location for init.d is different in my device
Click to expand...
Click to collapse
Yes. Probably there is none. What ROM are you using? Is it a Froyo ROM and from where?

howto apps2sd as ext4

Ok. So lots of talk about ext for apps2sd and how it's being mounted as ext2 even if you formatted to ext4 etc.
Also talk about how journalling hurts performance and such. We'll tackle all those points here.
1. Howto fix the mounting of your ext partition in the meantime till your mod updates their software.
A. Pre-requisites
You must have already partitioned your sd card with an ext4 fs for apps2sd. You can do this in recovery or some other app.
You should also get the program file expert. It's awesome. But any filemanager that has root ability will work. You can do all this in terminal too so whatever floats your boat. I'll use file expert below.
B. Remount your system fs as read/write (not sure if it's actually needed since /system is mounted rw by most mods already... but it can't hurt). You can do this in file expert by hitting menu and selecting More, then the top option labeled "Mount", select read write. It should say success.
You can also do this in terminal but like i said above, i dont think this is absolutely needed but i always assumed it was so never tried this not doing it.
C. go back down directories using the back key till you hit /. Then goto /system/etc/init.d/
D. select S05mountsd (the S0# isn't important here so if yours is different dont worry). Once selected, hit menu and select file operations. Select permissions and change permissions to read/write all.
E. Open the file with a text editor of choice. Scroll down the file to the part that has the command with -t auto in it. It should be pretty obvious, and it's near the end.
Change -t auto to -t ext4. It doesn't matter if your fs is actually ext2 or ext3, mounting as ext4 is always better. The ext4 block allocator and general code is more efficient even if you dont use ext4 features.
You may also want to add an option to the options list data=writeback. This disables journal caching of actual data and only caches metadata. This is really integral to performance on such a slow medium like sd cards and we'll discuss that later.
Save your file and exit.
F. Reboot your phone. You should now have your sd-ext mounted as ext4.
2. data=writeback and why?
there are multiple modes of journalling available to ext4. The default is ordered. This means that there is a specific order that information is written to the fs to ensure that nothing is lost that is said to be written. This means metadata about the fs and file data is both cached in the journal. Safe but slow.
data=writeback still allows crash protection but means you may lose actual newly written data. Your fs wont be corrupt, but your file may not have gotten those last updates you were writing and had not synced yet. It's a pretty decent compromise.
There are other forms that offer better performance but basically remove journal integrity and we're not concerned with those.
There are also options that determine the interval of journal syncing and you can find that information on ext4 man pages.
Why do we care so much about performance here? because sd flash is slow and finite. A journal file is like swap in that it is a static file that's contents get written to anytime you write to your fs. On a normal hdd, it's not noticeable really but on a class 6 SD card, you are going to notice having to write everything twice to your sd card. So we choose writeback mode to remove that double writing and only cache fs metadata updates so that the integrity of the filesystem isn't hurt, and only that particular file that we crashed prior to syncing gets effected. 99% of the time this is fine and perfectly safe and way better than nothing.
3. So if journalling can hurt performance by using up write bandwidth to a medium we are writing other stuff on, wouldn't swap be bad too?
Yes. Swap should only be put on sd cards if you absolutely need it or your phone wont work. Aside from stressing the card, the bandwidth to read/write is extremely small for something you basically have to be interactive with and move to and from ram. Additionally android complicates matters by managing memory at a higher level than the kernel and this means that swap is ignored as a portion of memory. So while technically userspace apps deal with swap transparently, the low memory manager of android will operate as if you didn't have swap...which defeats the purpose.
Use swap as a last choice.
Also, the ability to make swap partitions and swap files is a common issue in linux-land. It's best to use a partition as this removes fs overhead but it becomes even more of a concern when you put it on an ext4 fs with journalling active. With default settings, a swap file will be considered just another file and the journal will cache the data and metadata updates (very bad for performance). if you use swap on an ext4 fs that's actually mounted as ext4 you _NEED_ to make the data option writeback. You will be very disappointed if you dont.
4. Other important notes:
If you use any ext4 specific options like extents, you will no longer be able to read the files written by recovery programs that have not been updated to mount fs's as ext4. Also, just enabling extents doesn't convert data on the fs to use them. You have to write the files while the option is enabled to see their benefit. This is more easily done by putting the card in an actual linux computer and mounting with the correct options and copying the data from the filesystem and then back again ....not for the novice. Though any new files written will automatically have extents even if you dont redo all current data. So best to bother with that after a fresh install prior to restoring apps.
You may also be thinking... Why do i even need a fat fs, i can make the whole thing ext4 and it will be awesome! You would be right, it would be awesome. But it's not so easy. A couple things have to be done in order to make everything written world rewritable and accessible on such a partition. you would have to setgid to a group like "sdcard" to a directory in this new fs and give it rwx group permissions and then have all users current and future become part of that group and mount bind the directory to /sdcard. Needless to say, recovery apps would have to also be updated. Then you could make your main sd partition any fs your kernel supports.
Nice post, cm7 and diet gb I build already have -t ext3 set, do you think ext4 would make much difference?
ext4 has a better block allocator and other features both backwards compatible and ext4 specific. There's really no reason to not mount ext2/3 and 4 as ext4
Thanks for the answer, This is also in the kernel I compile,
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT23=y
CONFIG_EXT4_FS_XATTR=y
Yes, i began my work on the kernel tonight and noticed that in the latest kernels at least that ext4 is being masqueraded as ext2 and as i thought early on as might be the case, that the reporting of the partition as being mounted as ext2 is a bug that has been squashed in the newest vanilla kernels.
This is still an issue, but not a serious one. FS repair tools and such may bug out when trying to found out what fs they're dealing with as the kernel reports the wrong ext version but the functionality of the fs is not affected ( you still get journalling and all that jazz).
So this is less of a performance issue (cept the adding of data=writeback) and more of an issue with other programs needing to see the correct fs.
Ur Diet 09.05 still did not see ext4 partition by Titanium.
carz12 said:
Thanks for the answer, This is also in the kernel I compile,
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT23=y
CONFIG_EXT4_FS_XATTR=y
Click to expand...
Click to collapse
Sent from my Gingerbread on Sapphire using XDA Premium App
If the kernel you're using is one of the newer kernels (probably anything after 2.6.32...but you should be using something much newer) then it likely doesn't even have ext2 or ext3 compiled in. It's all just ext4 like it should be. The issue you still may have however is that the kernel reports the lower revision extfs because ext4's driver accepts the ext4 fs but the kernel is checking for ext2....
Basically, we are asking if this is an ext2fs. The ext4fs driver answers cuz it's all there is and it's masquerading as ext2 and ext3 and ext4. So it looks and sees ext4 and says, hey, i can read this and returns YES. Now the answer yes was to the question of ext2, so that's what gets reported everywhere, but internally, ext4 knows it's got an ext4 fs. mount options that normally would error out with -t ext2 are accepted because the fs driver isn't really ext2 and it knows the fs isn't either so the mount options pass.
This is a kernel bug in an aesthetic and practical sense. Functionally it's all working as intended but other apps and users will get confused as to what exactly is the filesystem, ext2, ext3 or ext4. It can't know without probing the fs itself. Which is stupid for many applications.
Supposedly the latest kernels have a patch to fix this to correctly report the correct fs revision of ext. Though perhaps that hasn't been pushed to linus's branch yet.
In any case, you still should be modifying the mount options to use data=writeback so you're not clogging up the little SD bandwidth you have with data caching in the journal. You really only need fs metadata caching so you dont bork the entire partition on a crash. data=writeback does that, but default is data=ordered.
thank you for such a good post!

[Script] swapconf - swap config

Description:
A small command line tool to create, delete, enable or disable swap file on sdcard and autostart swap file or partition on boot (zRam (compcache) is configured through system Settings))
Features:
- Automatic detection of partitions and files: if you do not have a dedicated swap partition, script uses swapping to file - first on sd-ext and if that one does not exist then it swaps to sdcard. Swap partition takes precedence over files. This should work no matter if you have or do not have ext partition (for app2ext) on sd card.
- create and delete swap file - you can test different sizes without rebooting
- ability to start and stop swapping where ever it may be configured: biggest advantage of this is that you can stop swap if you have it on your /sdcard so you can use mass storage mode. Also, you can test various sizes for swap files or how your system works when swap is disabled, without need for a reboot
- misc. activities: view status of swap partition/files and status of memory
Usage:
Code:
Disable startup: swapconf 0 -> disables startup script
Enable startup: swapconf 1 -> enables startup script
Create swap file: swapconf mkf size_in_MB -> creates swap file of specified size (e.g. swapconf mkf 30 for 30MB swap file)
Remove swap file: swapconf rmf -> removes previously created swap file
Start swap: swapconf start p|f -> starts swapping to partition (p) or file (f)
Stop swap: swapconf stop -> stops swapping whereever it may be started
View swap status: swapconf status -> view status of swaps and memory
NOTE: Enabling both swapping to partition and to file is not supported.
If you have a dedicated swap partition on sdcard it will be used if enabled. Otherwise you need to configure swap file.
The attachment is OpenRecovery flashable update zip. Simply apply update.
Startup script should persist across new ROM updates until you do a factory reset (wipe data)
Feedback is welcome
Changes for version 0.2:
- relocated script from /data/local/bin to /system/bin: if your terminal app does not export /data/local/bin to PATH that should not be an issue anymore. Script should still persist across ROM updates [thanks to kabaldan for the tips ], startup script should persist until you do a factory reset (or you delete/disable it ).
- reworked script commands: previously you were not able to create swap file without enabling startup script
- improved partitions detection
- fixed a bug where you could not stop swapping to file if zRam was active
- mics. fixes and cleanups
Changes for version 0.2b:
- fixed backup of swapconf script during ROM upgrade on CM10
Notes:
- script will not persist when changing different ROM versions, e.g. from CM7 to CM10
- kabaldan and Skrilax_CZ have informed me that it is a bad idea to have active swap on sdcard's FAT32 partition unless there is an app that will disable swapping when the sd card is going to be mounted to PC. So far, the only issue I have noticed is that you can not mount USB mass storage mode until you turn off swap so I left that possibility enabled. The choice is yours (btw., it uses fat32 partition only if you have no dedicated swap partition or ext partition).
Thanks a lot for the info.
Bravo mrvek!
"Svaka cast!"
Sent from my Milestone using xda app-developers app
Great piece of swap tool! Great jobs mrVek!
in my case i can't directly cmd 'swapconf status' as shown in your screenshot, but have to cmd 'sh swapconf status' than the script run...
Thank you anyway, save me lots of time.
@arttree: that's odd. I haven't had such issues. Do other commands have the same issue? What rom are you using?
That seems like a permissions problem, execute perm. maybe missing. I'll look into it. You can try to chmod 755 /data/local/bin/swapconf to see if it'll help.
/data/local/bin/swapconf permision is 755. i installed/flashed it using update-OR-swapconf.zip you posted in 1st post.
anyway other than execute using "sh", everything work fine, even after reboot my MS, swap partition detected automatically.
Do you have /data/local/bin in your PATH? Or you always navigate to /data/local/bin?
Try this:
echo $PATH
To see if /data/local/bin exists in your path, if not then:
export PATH= $PATH:/data/local/bin
after that you wont need "sh" or "./" before swapconf (and you will be able to execute swapconf cmd from everywhere)...
Remember that this is not permanent PATH export, you need to do that everytime, as mrvek guided me, in terminal's settings there is an option to execute some commands on console startup (similar to .bash_profile) and you should put there "export PATH= $PATH:/data/local/bin" so you don't have to do that every time on console startup.
Sent from my Milestone using xda app-developers app
great tips! It work! Thanks.
Little problem:
$ su
# swapconf 1
[[: /dev/block/mmcblk0p1: unknown operand
Created configuration.
Enabled startup script.
#
Seems like the script thinks that mmcblk0p1 is swap - but that's my ext-partition...
My swap is mmcblk0p2
But it seems like I'm misinterpreting this error, am I not?
Sent from my Milestone using xda app-developers app
Eiertschik said:
Little problem:
$ su
# swapconf 1
[[: /dev/block/mmcblk0p1: unknown operand
Created configuration.
Enabled startup script.
#
Seems like the script thinks that mmcblk0p1 is swap - but that's my ext-partition...
My swap is mmcblk0p2
But it seems like I'm misinterpreting this error, am I not?
Sent from my Milestone using xda app-developers app
Click to expand...
Click to collapse
mmcblk0 is sd card, if present. mmcblk0p1 is first partition and should be FAT32, mmcblk0p2 is second partition (ext, and it must be p2 because some things in system assume that ext is 2nd partition if it exists, otherwise it won't mount it) and mmcblk0p3 is 3rd (usualy swap).
Neverthelss, these are issues, especially on CM7, with the script that need to be fixed asap.
Updated OP with new version (0.2).
I have tested it with different configurations and it seems to be working. Let me know if I missed anything.
Thank you all for your feedback.
I hope not being nasty, more between your script and what we've used (-FuFu-v5), is there any difference important?
with this script you can config your settings by using the terminal
my script only aktivate swap and creates a swapfile (if needed), to config my script you must edit the script itself
Yes, interactivness.
ty!
bug fix
Updated OP (again).
Only change was a fix for CM10 so it should persist between rom upgrades (ie., swapconf script is properly backed up during CM10 install).
I'm sorry for the trouble, haven't noticed it before.
thank you for improving your scripts. Great piece of script that our life a lots lots easier.
btw, if i used swap file option, do i need to manually stop it before i enable the mass storage(connect to PC)?
arttree said:
thank you for improving your scripts. Great piece of script that our life a lots lots easier.
btw, if i used swap file option, do i need to manually stop it before i enable the mass storage(connect to PC)?
Click to expand...
Click to collapse
Yes but only if swap file is on sd card fat32 partition. If you do not have an ext partition then it was created in sdcard's fat32 (/sdcard/.swapfile)
If it is on your fat32 partition and is active you need to stop it before connecting to mass storage mode.
Please note that intended purpose of startup script is to enable swap on boot. To do that it uses certain priority - first it tries swap partition, if that did not exist then it tries ext partition (if you created file) and the last resort is sd card fat partition (also needs pre-created file). The same applies to manual creation of file - ext has precedence over fat32.
I guess that is not a design flaw.
is it fullly compatible with Compcache of zRam?
gambarini said:
is it fullly compatible with Compcache of zRam?
Click to expand...
Click to collapse
Hm... not sure if I understand.
zRam/compcache is controlled in ROMs settings.
On the other hand you can have zram enabled and still use this scrip to configure swap to partition or file but you can't manipulate zram with this script.

[script] 98swapon by -FuFu-

Hi there,
here now the official thread for my 98swapon script...
Latest Version: V11
Download: http://droid-world.bplaced.net/index.php?cat=Downloads&page=Scripte
there you will always find the latest version
Infos about the Script you can find here: http://droid-world.bplaced.net/index.php?cat=Milestone&page=Swap Aktivieren
but its German
here a translation using google translate (so please excuse the bad english, but i think my english is not much better then google translate)
Simply download the latest zip and copy to /sdcard/Open Recovery/updates and then run in the Open Recovery by apply update.
Best done previously a Nandroid backup of your current system, which you can import in case of problems again.
UPDATE
As of Version 9, there is on the SD card, the file swapon.conf with which you can adjust the different variables of the script, so it's no longer necessary to directly edit the script.
SWAPSIZE is the amount of swap in MB
If you put SWAPINT to 1, the internal memory is used for swap.
Swapdev determines which internal partition to use, possible / are cache and / data, all other partitions were too small to use it for swap.
RESETSWAP at 1 and at every reboot the swapfile is created, what is nützlch when changing SWAPSIZE.
SWAPIONLY set to 1 and it will only set the value for swappiness with boats, the rest are ignored.
For the rest of the values ​​consulted with Google, I can not say at all what may.
To edit the swapon.conf please use Notepad + + or it may cause problems.
Click to expand...
Click to collapse
i only test the script on cm7 and there it works fine...
if you have problems or find some issues, please let me know...
please post your swapon.log from /cache if you have a problem, and also your swapon.conf from sdcard.
changelog
V11
- add dos2unix to script, so that is now also possible to edit the swapon.conf with windows editor
V10
- add disable funktion to the script
- script now work on cm7 and cm10
Works perfectly on CM10.
Erovia said:
Works perfectly on CM10.
Click to expand...
Click to collapse
Did you try internal memory settings? It doesn't work for me on CM10.
main problem is, that the swapon.conf will not be found on boot...
so i need the mount points on bootup to see if the sdcard will be mount and for that i have line 19 in the script only thing to do is to delete the "##" to activate the line...
you also can edit the line directly in the zip befor apply it via OR apply update...
but if there still "/sdcard/swapon.conf not found..." in /cache/swapon.log there is something wrong...
may sdcard will not be mount on boot
or
there is no swapon.conf an sdcard
koodiifin said:
Did you try internal memory settings? It doesn't work for me on CM10.
Click to expand...
Click to collapse
Nope. I have serious concerns about using the internal memory as swap.
@Erovia
did you also have "/sdcard/swapon.conf not found..." in /cache/swapon.log?
if so i think its a mounting issues on cm10
I think you misunderstood me.
I have concerns about the idea of swapping on the internal memory, not problems with the script itself.
i understand you
i only want to know if the script found the swapon.conf on sdcard
Oh, in that case I misunderstood you.
You are right. It seems the config file is not found on CM10.
(Maybe if you use /mnt/sdcard/swapon.conf ? I know it is basically the same, I just dont have any other idea. Or maybe CM10 mounts the sdcard after the script runs? )
i add a mount command to the script, for cm7 it works fine but i think on cm10 it wont work...
but i dont kwno why...
for me it will be a help if you also delete the "##" on line 19 in the 98swapon script, the in the swapon.log all mountpoints will be shown....
on line 18 the script will try to mount the sdcard, but maybe on cm10 the mountpoint is different to cm7
i think i must flash cm10 this weekend to test it
I guess I found the source of problem.
It seems that sdcard is not being mounted. (although sd-ext is mounted)
on cm7 also the sdcard is not mountet on boot, first i try to mount with "mount /sdcard" but it wont work so i try "mount -t vfat -o rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,shortname=mixed,utf8 /dev/block/vold/179:1 /sdcard" and that works on cm7, after that the sdcard is mount on /mnt/sdcard and /sdcard
but i think that not work for cm10, but dont know way, maybe the sdcard will be mount but not on /sdcard.
for that i have add line 19 to the script to check that (for debugging its helpfull, but for release i comment out that line) so i have all mountpoint in the swapon.log to see what is mountet and where...
so i think i must flash cm10 to test it here to see where is the problem and its the fastes way for me
as i know the sdcard will not be mount on boot on no rom, cause there is no need to, the sdcard will be mount from android when the system is load
Wish I could help more, but I'm not really an expert in this kind of stuff.
An easy "workaround" would be if the config file was placed on the sd-ext as it is mounted properly. I know then it would be not as easy to edit it (for windows users) but it should work.
if i find no other way, i will add a option to use also /data and /sd-ext for the config file
at the moment i flash cm10 on my second device for testing
maybe its just a mountpoint issues
I think I found a solution.
I edited row #17 to this:
mount -t vfat /dev/block/vold/179:1 /sdcard >>$SWAPLOG
Basically I just deleted the parameters.
The log shows no error and /mnt/sdcard is mounted properly.
I ran some test and it doesn't work.
im testing
now i have 4 mount commands in the script an one of them work ^^ now i must find the one that work
i think i found the right command
maybe you can test it and confirme?
use
Code:
mount -t vfat /dev/block/mmcblk0p1 /sdcard
here it works, i reboot now 10 times and on every boot it mounts sdcard
if you confirm, i will update the script so that all can be test it ^^
now i just must test if it also works for cm7
Confirmed.
Works fine. :good:
Great job, again.
okay so now the latest version ist V10 ^^
i edit the first post and also uploaded the new script to my homepage
http://droid-world.bplaced.net/load.php?file=download/scripts/98swaponV10or.zip
it also works on cm7
now i have 1 device on cm7 and one on cm10 and btw, cm10 with 32mb swap on cache and sslauncher may works smooth so i stay on cm10 and test some days
Hey fufu I'm trying to enable internal memory swap with your new script. But now I'm getting a new error in the log.
Here's my config:
## 98swapon config file
##
## set to 1 to disable swap on boot
DISABLE=0
## set swapfile size in MB
## default setting is 32 for 32mb
SWAPSIZE=32
## set to 1 to use intern memory for swap
SWAPINT=1
## settings for use swap on intern memory
## use /cache or /data for swap
SWAPDEV=/cache
## recreate swapfile on every boot
## set to 1 to recreate swapfile on every boot, may be helpfull if you change SWAPSIZE
RESETSWAP=0
## set to 1 to set only swappiness value all other values will be ignored
SWAPIONLY=0
## vm settings
## swappiness
SWAPPINESS=15
## dirty radio
DRADIO=80
## dirty backgriound radio
DBRADIO=30
## vfs cache pressure
VFSCP=10
## dirty expire centisecs
DEC=1000
## dirty writeback centisecs
DWC=1000
## laptop mode value 0 or 1
LAPTOPM=0
## oom kill allocating task value 0 or 1
OOMKAT=1
## panic on oom value 0 or 1
POOOM=0
Here's the log:
please set DISABLE in swapon.conf to 0 or 1
swap disabled in case of wrong DISABLE value
Sent from my Milestone using xda app-developers app

Categories

Resources