since thats where the phone storage is at can we just move it to like system/sd/app ??
would that cause any problems,
Actually the variable phone stores are in /data (mtdblock5), not /system (mtdblock3).
While it's technically possible to move all teh files and have /data be mounted from a partition the SD card, there wouldn't be any real gain.
/data contains a lot of caches and other files that are used constantly by the system - these need to be accessed quickly for the phone to be responsive - this is why we only move portions of this directory.
There would be even less benefit to moving mtdblock3 (/system)
Since the guts of the OS are in that partition, your phone would always be reading from the SD card, reducing battery life somewhat, and slowing things down.
The interface to the internal flash memory bus is considerably faster than the SD bus - or at the very least there's considerably less latency - even a class 6 SDHC card pales in comparison. Apps (which are unpacked and loaded into RAM), can live here well since they're only really read once when you spawn the app.
System libraries and caches on the other hand are much happier on the internal flash memory. Some ROMs do move the cache files in /data, but it seems to often lead to trouble, and no real gain in performance.
Is there a cogent reason why the SD card cannot be partitioned without a FAT32 filesystem? From all the benchmarks I've ever seen, EXT4 kills FAT32. However, when I tried using an SD card partitioned with only an EXT4 partition, vold (the volume management daemon) wasn't happy at all.
I can manually mount the volume from the shell, but that does no good as far as android recognizing that the partition has been mounted.
I'd be willing to hack up the vold source code if it would be useful, but I just want to know if anyone has any thoughts or experience on the issue.
This question has been asked in the past, but no one has every replied to it. Is there no one with experience in this area?
Thanks
bump........
That's just what is most common and what it looks for. You could try and look at the apps2ext scripts for mounting etc.
What real performance gain could you see by using just ext4 in this scenario, my guess it nothing.
There's possibly no speed difference, though benchmarks with flash do show ext4 to be faster. The bigger point though is why have both an ext and fat partition, when you can have only one?
Also, I don't see why this should be an issue. 'mount' can auto-discover the filesystem type of a partition, so the only reason this shouldn't already "just work" is because vold is forcing it to be FAT. I find this ironic given that the main system partitions are actually YAFFS2.
Information
JIT Compiler
Going Deeper With Android 2.2′s JIT Compiler
by Quentyn Kennemer on May 26th, 2010 at 3:09 am
Before the official Froyo announcement at I/O, we’d learned that an Adobe employee’s phone was running at ridiculously high speeds. Most outlets quickly chalked this up to the possibility that Google implemented a Just-In-Time compiler for their Dalvik Virtual Machine.
Sure enough, they confirmed our suspicions at the keynote in San Francisco, and we even got a taste of real-world performance (there’s no “theoretical” performance hikes here, folks). We know what JIT does, but I’m sure there are a lot of non-developers out there that can’t understand how it’s able to provide the huge bump in performance for Dalvik.
Technical lead for Android’s Dalvik team – Dan Bornstein – signed in with a blog post over at the Android developers site going into a bit more details about what’s really going on beneath the hood to significantly improve performance of Android (in certain cases) without needing to touch the hardware. Thankfully, he puts it in plain English so you can get – at the least – a pretty basic understanding of what’s going on. Head over there now if you want to learn more about what makes your Froyo so sweet.
Linux-Swap
What is SWAP?
Swap space is an auxiliary storage, such as a portion of a hard-disk, which can be used as memory by the operating system when system RAM is insufficient. This is especially useful on systems with very little system RAM, such as most DD-WRT compatible routers, as it helps prevent the system from running out of memory when multiple background processes are installed.
The difference between Froyo A2SD, A2SD and A2SD+
One of the glories of using Android is having an SD card for storage rather than having internal-only memory. Thanks to Android’s Linux blood, you can even harness the power of the SD card beyond its usual file storage capabilities. On Android, your internal memory is precious and you do not want to install games and applications – especially those which are up to 50MB in sizes – on your internal memory just to make your phone run slower. Instead, you can install the applications on SD card and let your phone take care of the system apps instead, saving more internal memory and speeding up your phone.
The Partitions
Android natively supports fat32 partition. However, thanks to the Android community, support for swap and ext partitions can be enabled too. Depending on the ROM, some can support up to ext4, while others support up to ext3. The explanation about the differences between these partitions is indeed very lengthy and not part of our chapter today. What is important to know, is that Android has support for fat32, swap, ext2, ext3, and ext4 partition support.
Your SD card is by default formatted to fat32. In order to use swap and ext partitions, you need to repartition your SD card. There are several ways to do this, but the most common way is by using a custom recovery installed on your phone (ClockworkMod or AmonRA recovery). Swap is virtual memory which uses extra space on your SD card for virtual memory. However, since Android already has DalvikVM, swap is not really needed. I myself don’t use swap space on my SD partitions. Ext partition is extended partition which was the first ever type of partition created specifically for Linux. It is based of the standard UNIX file system and was designed to overcome the limitations of Minix file systems. Ext 2 is second extended partition, ext3 is third extended partition and ext4 is the fourth extended partition respectively.
Dalvik Virtual Machine
One of the best functions of Android has to be the Dalvik cache. Dalvik cache is a wonder from the point your Android starts up, runs, hibernates and all the way till you device shuts down. Dalvik cache collects the information about the installed applications and frameworks, and organizes them into a writeable cache. Under this writeable cache, it stores the “optimized” bytecode of the applications which is used by the applications themselves later for a smoother operation. This dalvik cache can grow immensely huge as more applications are installed on your phone. It is safe to wipe dalvik-cache. It will be rebuilt again when the phone boots. This also explains why your phone takes ages to start up for the first time. As for my Nexus One, having about 145 applications installed, it takes about 13 minutes to build the cache.
If you ever extract an APK installer file, you will always find a file named classes.dex. This is the file Dalvik finds to build the cache. What makes the process slow? APK is an archive (which is why you can open it up with an unarchiver such as WinRAR or 7-Zip). Being an archive, it provides limited write access to the files contained within and the fact that archives are compressed. Not to forget, APKs are encrypted archives too. Therefore, DalvikVM has to extract the classes.dex files and build the Dalvik table accordingly which makes it easier to write data on it too. With this collective set of data, the Android OS no longer needs to index the applications and find their classes.dex when the phone is already running. Instead, it will just look into one place, and will know what to do next. Nifty huh?
To know what is going on inside the Dalvik VM, you can read about it here.
Froyo A2SD (F-A2SD)
When Froyo was released to Android community, one of its new features was the A2SD implementation. F-A2SD uses fat32 partition natively for application storage. This means, all you have to do is just slot in your SD card and its all ready to go. There is no need to partition the SD card whatsoever. This was a great effort from Google to include A2SD to the Android OS as it gives you an option to choose which applications you want to move to SD card, and which you want to leave on internal memory – BUT – with a condition! If the application developer decides to protect his application and not include A2SD support, you would not be able to move it to SD card. One major problem that F-A2SD has is that it only uses fat32 partition. This way, when the SD card is mounted to the computer, the applications become inaccessible. Not only that, if the applications have widget support, the widgets are removed too when the SD card is mounted. This can become rather a hassle especially if you have to mount your computer several times in a day.
A2SD
The A2SD method is much more interesting. It harnesses the glory of ext partitions. This way, the applications (protected or non-protected), will all be installed on the SD card ext partition. The good thing about ext partition is that when you mount your SD card, the ext partition is NOT mounted together. This said, when your SD card is mounted, the applications will still be accessible and separated from the files and folders on your fat32 partition. On A2SD, the dalvik cache resides on the phone memory.
A2SD+
A2SD+ takes the A2SD one step further. Its pretty useless to have a 512MB A2SD capacity if your dalvik cache is still on phone memory and you have lots of applications installed. This is because the dalvik cache can become pretty huge and just by using HALF of your A2SD’s ext partition, your internal memory can become FULL because of dalvik. Therefore, in A2SD+, the dalvik cache is also moved to SD card. This way, your internal memory is free as a highway. However, remember that Dalvik cache is accessed very frequently. If you have a slow SD card, the overall performance might be affected. I recommend using a class 6 or class 10 SD card for the purpose.
So make your pick guys. In Android’s world, you always have choices. Depending on your needs, use the partition that satisfies you. Hope this article clears any misunderstanding that anyone might be having. Cheers~
Filesystems (post is for galaxy s but all android is ~ same in filesystems)
Reality behind RFS Lag
Background
All data is stored on an 8gb or 16gb MoviNAND chip, of which 2GB is ‘system data’, and the rest is for user storage. The MoviNAND is one of the first mobile ‘smart SSD’ chips. That means that the MoviNAND handles all operations such as data wear leveling, physical data lookup, as well as having it’s own internal buffers. This cleverness is both good… and very bad.
RFS
RFS has a fairly badly written driver, that will call an fsync on file close.
Basically, RFS runs in ‘ultra secure’ mode by default. This security may not be really needed – I personally don’t want it if it means enormous slow downs. It also doesn’t help data security if the system/app is holding a file open, only if it closes the file. The MoviNAND is also fairly smart, and appears to write it’s cache to disk before turning off, and also appears to have capacitors to keep it alive for a little bit of time in the event of a power cut.
SQLite
Most Android apps use SQLite – a fairly simple database that is easy to embed. Sqlite has ‘transactions’ – not real transactions, but a transaction in sqlite is where the database is locked for the duration of a database write, and multiple databases writes can be included in one transaction. At the end of a transaction, sqlite will call FSYNC on the database file, causing a possibly long wait while the MoviNAND does it’s thing. Certain applications will not bunch up writes into a single transaction, and will do all of their writes in new transactions. This means that fsync will be called again and again. This isn’t really a problem on most devices, as fsync is a very fast operation. This is a problem on the SGS, because MoviNAND fsync is very slow.
The various fixes and why they work
Native EXT4 to replace RFS (Voodoo)
By replacing RFS with EXT4, the ‘sync on fileclose’ problem is removed. The EXT series of filesystems is also more efficient at allocating information into blocks than RFS/FAT32 is. This means less real writes to MoviNAND, which means that the MoviNAND buffer should be smaller, and when a sync is called, fewer commands have to be run. When a sync is called on EXT4, it will still be very slow, as the MoviNAND’s sync is still slow.
Basically, EXT4 improves filesystem grouping which leads to less commands, and does not have the broken ‘sync on file close’ that RFS does. It will not heavily improve sqlite database access in certain apps, as the full fsync on transaction end will still have to go through MoviNAND, and will be slow.
When pulling out the battery, there is a chance to lose data that has been written to a file but has not yet been told to sync to disk. This means that EXT4 is less secure than RFS. However, I believe the performance to be worth the risk.
Loopback EXT2 on top of RFS (OCLF)
By creating a loopback filesystem of EXT2, the ‘sync on fileclose’ problem is removed as well. Since the Loopback File is never closed until the EXT2 is unmounted, RFS will not call fsync when a file in the EXT2 loopback is closed. Since a single large file is created on RFS instead of multiple small files, RFS is unable to mis-allocate the file, or fragment it. The actual allocation of filesystem blocks is handled by EXT2. As a note, care should be taken in making the large file on RFS – it MUST align correctly with the MoviNAND boundries, or operations will be slowed down due to double-disk accesses for files, etc. It is unknown whether OCLF is aligning this correctly (how to determine this? 4KB block size gives double the performance of 2KB block size, so it might be aligning it correctly already).
Loopback also has the benefit of speeding up Sqlite databases (at the expense of a transaction being lost in power outage, as it could still be in ram). As always, this is a performance tradeoff between data security when the battery is pulled out, and performance. When pulling a battery out while using the loopback filesystem, there is a chance to lose the last few seconds of database writes. In practice, this isn’t a huge deal for a mobile phone – most lost data will be resynced when the phone reboots. In my opinion, the performance is worth it because of the very slow speed of a sync on MoviNAND.
Loopback EXT2 on top of EXT4
All of the above for normal loopback EXT2 applies. In addition, when the loopback flushes data, it will be flushed to EXT4 instead of RFS. This will probably be better than flushing to RFS, as the RFS driver is not as well written as the EXT4 driver. The difference should not be very large, though.
Journaling
Journaling on an SSD is not required. Your data will not be lost, your puppy will not die. Here is a post made by Theodore Tso -http://marc.info/?l=linux-ext4&m=125803982214652&w=2
But there will be some distinct tradeoffs with
omitting the journal, including possibility that sometimes on an
unclean shutdown you will need to do a manual e2fsck pass.
Not using a journal is not a big deal, as long as you take care to do a full e2fsck pass when an unclear shutdown has occurred. This is the main reason for a journal – to prevent the need to do a full disk check, and instead the journal can be easily read, and the full disk check avoided.
EXT2 vs EXT4
EXT2 appears to work better on the SGS than EXT4. This is because EXT4 has more CPU overhead than EXT2. Journaling is also very bad on MoviNAND. Why? It appears to be the command buffer in the MoviNAND controller. A call to update the journal will use a command slot in the MoviNANDs buffer, that could otherwise have been used for a real disk write. This means that journaling on MoviNAND is a VERY expensive operation compared to journaling on a ‘dumb’ disk.
Well, you could technically use EXT4 and simply disable the high cpu and other features until you are left with EXT2, since EXT4 and EXT2 are basically the same thing.
At any rate, the difference between EXT4 and EXT2 is not very large, and there’s no need for flamewars over it – it comes down to a choice of ‘running’ performance vs ‘startup’ performance, with EXT2 edging out EXT4 for everyday speed, while EXT4 not required a long disk check at boot.
Future Work
Rewrite the firmware for the MoviNAND’s flash to handle fsyncs properly and not bring the system to it’s knees. I joke, but this is really the true solution.
Other solutions include hacking EXT’s fsync method to return instantly, and ensuring that the real fsync is called when the system shuts down. Or doing nothing, fsync is there for a reason, I guess, and would be fine if MoviNAND’s fsync wasn’t so very slow.
There is probably a lot of small details missing from this writeup. They’ll be updated when we learn more. Thanks for all the useful discussions and arguments, everyone!
Many Thanks to Dennis for this awesome information (officially posted here)
Nice cp, but you could take out everything about movienand, which is not present in g3 we only have onenand...
Ah, and don't forget to give credits or link to op...
FadeFx said:
Nice cp, but you could take out everything about movienand, which is not present in g3 we only have onenand...
Ah, and don't forget to give credits or link to op...
Click to expand...
Click to collapse
Of course Done.
kyrillos13 said:
Of course Done.
Click to expand...
Click to collapse
hi
very useful info for the members very nice of u added to the roll up thread
I don t know why I never read this. Very nice and interesting info.
thank you very much for this!!
What about the link2SDapp? Is this a good choice? I'm using it but think it has probs with the second partition.
Godyn said:
What about the link2SDapp? Is this a good choice? I'm using it but think it has probs with the second partition.
Click to expand...
Click to collapse
Better use app2sd scripts
Thx,
I used it, but it filled my ROM.
What app do you suggest?
I used app 2 SD https://market.android.com/details?id=com.a0soft.gphone.app2sd&feature=search_result
But that didn't solve it.
I'd like to search this forum for help, but am uneducated enough on the internals of the android OS that I'm not sure what to search for.
i'll ask my question expecting an answer on either (1) how to search for such information in the future, or (2) how to solve the issue.
I installed a 16GB class 10 SD card in my transformer keyboard and would like to edit the appropriate config files so that all future software installs and downloads are on that storage card rather than the internal 16GB memory. So my question is what files do i edit to accomplish this?
thanks...
marvin
You want Apps2SD. Unfortunately, unless something has changed recently, you can't use it on your TF.
www.transformerforums.com/forum/asus-transformer-help/3514-tf-not-letting-me-move-apps-sd-save-things-sd.html#post27629
mncessna said:
I'd like to search this forum for help, but am uneducated enough on the internals of the android OS that I'm not sure what to search for.
i'll ask my question expecting an answer on either (1) how to search for such information in the future, or (2) how to solve the issue.
I installed a 16GB class 10 SD card in my transformer keyboard and would like to edit the appropriate config files so that all future software installs and downloads are on that storage card rather than the internal 16GB memory. So my question is what files do i edit to accomplish this?
thanks...
marvin
Click to expand...
Click to collapse
I'd probably try searching for "Move apps to SD" if I was searching for that topic(I know it's hard to figure out the right search terms sometimes, but by posting in Q&A, you've at least posted a fair question in the right place), here's my answer in two parts:
1. I don't think Apps2SD would work with the Dock SD slot.
2. You have the entire 16/32GB internal storage shared between apps and the /sdcard mount. Is that really not enough space?
Also, the internal storage is faster than a Class 10 card, so there will be a performance hit running from SD.
thanks...you've given me food for thought
2. You have the entire 16/32GB internal storage shared between apps and the /sdcard mount. Is that really not enough space?
Click to expand...
Click to collapse
there's never enough space!! I've always attempted to offload temp files, downloads, and various other files from the same partition as the OS. prevents excessive fragmentation etc. i'm assuming this would still be an issue even with non-spinning storage, maybe it isn't?
Also, the internal storage is faster than a Class 10 card, so there will be a performance hit running from SD.
Click to expand...
Click to collapse
now that's an interesting comment. i thought Class 10 was the fastest AND that internal storage on the transformer was the same technology. Hmmm.....more research for me.[/QUOTE]
thanks for the information and advice.
marvin
mncessna said:
there's never enough space!! I've always attempted to offload temp files, downloads, and various other files from the same partition as the OS. prevents excessive fragmentation etc. i'm assuming this would still be an issue even with non-spinning storage, maybe it isn't?
now that's an interesting comment. i thought Class 10 was the fastest AND that internal storage on the transformer was the same technology. Hmmm.....more research for me.
thanks for the information and advice.
marvin
Click to expand...
Click to collapse
Theoretically, fragmentation can slow a flash drive down, but I've never known anybody to notice the difference or to bother with defragmenting regularly on a flash device. Maybe somebody here is into that, but it's just not practical IMHO. Apps2SD is for people who run out of internal space and need MORE room for apps. If you are pushing 12gb (the ballpark usable space on the internal memory of the TF) of apps, you need to get back into the work force. You're better off trying to keep large data files on the removable drive first, even if it means just relocating them every once in a while.
The bottom line is, the way the Transformer treats the built-in memory as an SD card is going to be an issue when trying to use an external card for Apps2SD (which I still haven't heard of anybody succeeding at), but you may be able to set your default download folder to the removable card. If you're dealing with a browser, it would depend on which one. If you're doing your own file transfers, just get a file manager that allows you to set your "home" directory and point it to the /Removable/MicroSD directory.
SilverZero said:
Theoretically, fragmentation can slow a flash drive down, but I've never known anybody to notice the difference or to bother with defragmenting regularly on a flash device. Maybe somebody here is into that, but it's just not practical IMHO.
Click to expand...
Click to collapse
Even on hard drive it's most of the time not worth the effort. On SSD just forget about fragmentation.
i thought the reason fragmentation is a performance hit for HDDs is due to the fact it has to spin to get to the rest of the file.
SSD does not need anything like that.
fragmentation causes delays
finalhit said:
i thought the reason fragmentation is a performance hit for HDDs is due to the fact it has to spin to get to the rest of the file.
SSD does not need anything like that.
Click to expand...
Click to collapse
a disk read consists of a seek op to place the read head over the correct cylinder, then rotational delay while the correct sector rotates under the head, then a data read, and then a transfer to core memory. when a file is fragmented, this operation is repeated multiple times in order to read the entire file into memory. disk drives are pretty fast these days, but they are still the slowest part of most modern computers. cpus and memory are very fast, but disk drives slow things up, therefore, adding to io delays with fragmented files is a real performance hit.
however, i have no data on how fragmentation affects flash memory...perhaps not at all?!.
thanks to all for inputs.
Here's one perspective on flash fragmentation:
www.wizcode.com/articles/comments/flash_memory_fragmentation_myths_and_facts/
There seems to be quite a bit of discussion about this but no real resolution. I've been trying to find a way to mount directories on my external SD card onto the internal /sdcard directories to force apps that only can write to /sdcard internally to actually write to the external card. What a mess. I've tried symbolic links, battled the FUSE mounts, experimented with "mount -o bind" -- on and on. FolderMount claims to do such things, but bricks the phone.
I can do the bind mounts -- and when I look at them with adb they appear to be correct. But the apps continue to write to the internal /sdcard dirs. I've also turned off SuperSu mount namespace separation in testing.
It's a shame that something that really should be so simple is such a PITA. Any new ideas? Thanks!
Why don't you use ADB and partition your SD card as internal storage. Samsung took out the adoptable storage feature for Marshmallow. For the workaround go here:
http://forum.xda-developers.com/galaxy-s7/how-to/enable-adoptable-storage-s7-t3333511
I considered that early on. The problem is that I've never seen a reasonable explanation of how "bonded" memory like that is handled. Right now I have a 64GB card split as one exFAT and one ext2 partition (the latter for link2sd, and mostly empty). But at least this lets me have some control over where apps end up. But if I bond ("adaptable") an SD card partition to the internal memory, I (AFAIK) lose that control. Any given app and app's data could end up on faster internal memory or slower external memory, and I wouldn't be able to control it. Also, at this point wiping the phone to do this would be a serious PITA. Am I missing something? Thanks!
Actually, you can partition part of the SD card as internal and part of it can remain external using the ADB method. I have a 64GB card myself, and it works great. I would recommend a high speed one though.