I've seen a lot of people complaining about not being able to boot after using A2SD for a while and I think it's definitely necessary to make clear the danger in using A2SD and some ways to prevents them.
Do not move /data/data to SD.
You can safely move /data/app and /data/app-private to SD. Be cautions with /data/dalvik-cache. (See below)
Make sure the ext2 partition are mounted with noatime and nodiratime (rom maker's job)
Do regular file system checks (using Linux)
Back up your ext2 partition and redo them regularly
Here're the theories:
(NAND) Flash memory has two serious limitations when used to store frequently changing data: you cannot do random write unless you erase a whole page first and any bit can only be written a limited number of times (typically a couple millions). When one bit in a page is detected to be faulty, you lose the whole page. (More details: http://en.wikipedia.org/wiki/Flash_memory#Limitations )
Most mobile os vendors overcome this problem by using special file systems, namely yaffs and jffs, which arrange files according to page size and do write operations only when necessary. They also provide journals so hardware faults could be reliably detected and corrected.
EXT2, which is the file system used in A2SD, DO NOT have these features. It may stupidly allow several pages to be erased hundreds of times of just to write some small files into it. Things are made even worse because some fixed part of the file system (inode table and bitmap) has to be written EVERYTIME a file operation is done. The will accelerate the wearing of those pages and when they become inaccessible, you lose the whole file system.
A2SD would work fine if you only move /data/app to it, because the application files are never modified. It is a completely different story for /data/data because the sqlite databases in there are modified almost every other second!! (And I suspect the OS commit them to disk very often to ensure data integrity.)
In addition, android does not have system check tools for ext2, so it will not be able to detect any problem with the file system until it’s too late.
Edit:
I am not sure how frequent the system updates files in dalvik-cache, but I would say you only move it when you are running out of space in /data.
(Also changed title)
billc.cn said:
I've seen a lot of people complaining about not being able to boot after using A2SD for a while and I think it's definitely necessary to make clear the danger in using A2SD and some ways to prevents them.
Do not move /data/data to SD.
Do regular file system checks (using Linux)
Back up your ext2 parition and redo them regularly
Here're the theories:
(NAND) Flash memory has two serious limitations when used to store frequently changing data: you cannot do random write unless you erase a whole page first and any bit can only be written a limited number of times (typically a couple millions). When one bit in a page is detected to be faulty, you lose the whole page. (More details: http://en.wikipedia.org/wiki/Flash_memory#Limitations )
Most mobile os vendors overcome this problem by using special file systems, namely yaffs and jffs, which arrange files according to page size and do write operations only when necessary. They also provide journals so hardware faults could be reliably detected and corrected.
EXT2, which is the file system used in A2SD, DO NOT have these features. It may stupidly allow several pages to be erased hundreds of times of just to write some small files into it. Things are made even worse because some fixed part of the file system (inode table and bitmap) has to be written EVERYTIME a file operation is done. The will accelerate the wearing of those pages and when they become inaccessible, you lose the whole file system.
A2SD would work fine if you only move /data/app to it, because the application files are never modified. It is a completely different story for /data/data because the sqlite databases in there are modified almost every other second!! (And I suspect the OS commit them to disk very often to ensure data integrity.)
In addition, android does not have system check tools for ext2, so it will not be able to detect any problem with the file system until it’s too late.
Click to expand...
Click to collapse
these are very good points. Actually come to think of it we better mount the ext2 partition with noatime. Because right now every read will wear the flash down. flash storage is really not meant for ext2 filesystem.
knaries2000 said:
these are very good points. Actually come to think of it we better mount the ext2 partition with noatime. Because right now every read will wear the flash down. flash storage is really not meant for ext2 filesystem.
Click to expand...
Click to collapse
In JF1.5 build it is mounted with noatime, i believe:
#mount
/dev/mmcblk0p2 on /system/sd type ext2 (rw,noatime,nodiratime,errors=continue)
But I totally agree with the point of the thread /data/data should not be moved to sd. Not that it's only dangerous (as described), I even don't see any advantages of it.
Dimath said:
In JF1.5 build it is mounted with noatime, i believe:
#mount
/dev/mmcblk0p2 on /system/sd type ext2 (rw,noatime,nodiratime,errors=continue)
But I totally agree with the point of the thread /data/data should not be moved to sd. Not that it's only dangerous (as described), I even don't see any advantages of it.
Click to expand...
Click to collapse
really. that's good, but I am running haykuro's build right now and it is not mounted with noatime. I will have to change the init script.
I'm using JF's 1.5 A2SD build, and I'm pretty certain I moved over /data/data. I didn't really ask myself the question when doing it, but what exactly is stored in /data/data? Is there a command I can run to move it back off the SD card?
Are ext2 and fat the only supported file systems in the android kernel? If not maybe it would be best to move to a wiser file system.
Rekna said:
Are ext2 and fat the only supported file systems in the android kernel? If not maybe it would be best to move to a wiser file system.
Click to expand...
Click to collapse
Yeah, why can't we use yaffs etc?
Dimath said:
Yeah, why can't we use yaffs etc?
Click to expand...
Click to collapse
i don't think most partitioners support yaffs
i know the partition manager on ubuntu 8.10 doesn't
tubaking182 said:
i don't think most partitioners support yaffs
i know the partition manager on ubuntu 8.10 doesn't
Click to expand...
Click to collapse
Indeed, and I'm fairly sure that's why it hasn't been done. But keep in mind several million write cycles is a heck of a lot and apps only really write their data caches occasionally, so it'll likely be a while before anything bad happens(on the scale of years) so this is slight sensationalism. Actually the term "cache" is a slight misnomer here since it's really just storage the apps use for temporary data. If it were used as some kind of extended RAM or a real cache then I could see problems but with what it's used for it should be a non-issue.
billc.cn said:
Here're the theories:
Click to expand...
Click to collapse
SD cards are a form of removable flash that have their own write controllers. In most cases, the write controllers also perform wear levelling. This means even if a program writes to the same file on the sd repeatedly, each time it writes, it is not writing the same physical location on the flash. The reason for this is because due to the way flash works, entire blocks have to be erased before they can be rewritten. To make writing faster, the memory controller keeps a list of empty blocks that are ready to use. When a file is changed, the entire file with the new changes is written to a new memory block and then the old block with obsolete data is then reset to zeros (data deleted).
That said, I still think A2SD is a red herring that only contributes to newbies spamming these forums.
jashsu said:
SD cards are a form of removable flash that have their own write controllers. In most cases, the write controllers also perform wear levelling. This means even if a program writes to the same file on the sd repeatedly, each time it writes, it is not writing the same physical location on the flash. The reason for this is because due to the way flash works, entire blocks have to be erased before they can be rewritten. To make writing faster, the memory controller keeps a list of empty blocks that are ready to use. When a file is changed, the entire file with the new changes is written to a new memory block and then the old block with obsolete data is then reset to zeros (data deleted).
That said, I still think A2SD is a red herring that only contributes to newbies spamming these forums.
Click to expand...
Click to collapse
Which is exactly why I made my new method that's able to deal with a lot of user mistakes and can be incorporated into ROMs to make it take almost 0 user effort
Dimath said:
Yeah, why can't we use yaffs etc?
Click to expand...
Click to collapse
yaffs on SD cards can in some common cases invalidate the wear leveling in hardware that SD cards do, as they so the wear leveling in software.
You are simply understimating ROM makers, SD cards are different from the internal flash in that they do auto wear leveling, that's why you can put common filesystems like FAT which have statically placed allocation tables and writes to the same logical sector will always land on very different places in the card every time.
Yes, noatime will help a lot as it will _reduce_ writes to your SD card.
ext2 is not journalled so it will have less writes too than ext3 or any other journalled filesystem.
So ROM makers are already doing a good job, don't understimate them please.
Im trying out a theory. I running JAC Hero 2.3 and wanna try running the swapfile on the /data partition. I moved app_s over to /system/sd and used swapper app to create the swapfile but android doesnt seem to use it. when i run free It says Total/used/free are all 0 but the swap file is 34 meg. I tried putting it in a sub folder and chown root.root on the folder before setting up and still no luck. Anyone have any ideas why it wont work on /data or how to get it to work on /data?
Once you run swapper go into the setting and change the location of your swap file from /sdcard/swapfile.swp to /system/sd/swap.swp ( see sxfx post[url]
you don't want to do that. Swapping involves a lot of writing and erasing, it'll wear out your internal chip and you'll start experiencing reduced capacity, write cycles for nand are even less than for flash memory!
jubeh said:
you don't want to do that. Swapping involves a lot of writing and erasing, it'll wear out your internal chip and you'll start experiencing reduced capacity, write cycles for nand are even less than for flash memory!
Click to expand...
Click to collapse
not to mention, swapon as implemented (on every build i've tried) doesn't work on a swapfile stored on a yaffs2 partition
jubeh said:
you don't want to do that. Swapping involves a lot of writing and erasing, it'll wear out your internal chip and you'll start experiencing reduced capacity, write cycles for nand are even less than for flash memory!
Click to expand...
Click to collapse
Actually /sdcard/ is your fat32 portion of your sd card, /system/sd/ is the ext2/3 of your sd card.
Also I have done a bunch of research on this write/read fiasco just because of android.
And even if you set a swap file to your SD sure it will shorten your life of the card but it will still last you at least 2 years.
I have been using USB devices on linux as swap locations forever now and I still have thumb drives that have been used and abused for months and months as a swap place and they are still pulling strong.
As for the internal chip, Im not sure what you mean by that?
Even if you could put swap on the internal flash, its not going to be faster.
Putting swap on the internal flash will make things slower as the internal flash is about 3 to 5 times slower than a class 6 sdcard.
you're comparing using a flash drive for swap in a full blown computer that probably has around 1-4gb of ram, the swap file is hardly ever touched, unless you're running a lot of applications at a time. Dream has only 90 mb available to dalvik, and rosie is a big fat... lady... plus linux/dalvik manage memory in a different way, so files are often dropped to swap and they dont stay there for long (maybe in a 256 swap, but not in a 32 mb one).
the OP also is talking about moving his swap to his internal storage (chip, nand, whatever), as he thinks it's having no effect working from the sd card. Personally, i think that both a2sd and swapper are flawed. They're overcompensating for an os that was not meant to run on that device, and the real work should be in porting (as in developing, not just file-swapping as most "devs" do here) a launcher app that we can feel comfortable with running on a stock android system with stock (or slightly improved) libraries and that we can call comparable to rosie. Just look at ahome or dxtop or openhome, they're good, solid, great looking home replacements that work as well or better than rosie, but they run out of the stock libraries. I wonder why nobody has made a free, open source home replacement app yet
jubeh said:
you're comparing using a flash drive for swap in a full blown computer that probably has around 1-4gb of ram, the swap file is hardly ever touched, unless you're running a lot of applications at a time. Dream has only 90 mb available to dalvik, and rosie is a big fat... lady... plus linux/dalvik manage memory in a different way, so files are often dropped to swap and they dont stay there for long (maybe in a 256 swap, but not in a 32 mb one).
the OP also is talking about moving his swap to his internal storage (chip, nand, whatever), as he thinks it's having no effect working from the sd card. Personally, i think that both a2sd and swapper are flawed. They're overcompensating for an os that was not meant to run on that device, and the real work should be in porting (as in developing, not just file-swapping as most "devs" do here) a launcher app that we can feel comfortable with running on a stock android system with stock (or slightly improved) libraries and that we can call comparable to rosie. Just look at ahome or dxtop or openhome, they're good, solid, great looking home replacements that work as well or better than rosie, but they run out of the stock libraries. I wonder why nobody has made a free, open source home replacement app yet
Click to expand...
Click to collapse
You make a very good point about computer having 1-4gigs of ram and not needing a swap file.
Honestly I almost never run a swap file on a computer that has more then 1gig of ram.
Plus90% of the time when I do run a swap file is when Im running a live distro of linux of a cd, but also 90% of the time I run the distro from the flash drive instead of cd cause its much faster.
So now imagine how many reads/writes I abuse that flash drive with by running a full OS on it.
Also I don't understand how a2sd or swapper is flawed? Just because they do their job? I mean it's not really our fault that they made the G1 with a little less memory spaces then we would like it to have.
But that's exactly why we have tools like swapper and a2sd. Plus no one is really forced to run these roms on the G1 phone and those of us that do realize that we have to take extra steps in creating tools to help it.
And that's not only true for the G1 but anywhere in the computer world these days.
dwang said:
Even if you could put swap on the internal flash, its not going to be faster.
Putting swap on the internal flash will make things slower as the internal flash is about 3 to 5 times slower than a class 6 sdcard.
Click to expand...
Click to collapse
I'd like to spread swap over the sd and internal storage if possible -- should make paging a lot less evident if priorities are set up properly.
dwang said:
Even if you could put swap on the internal flash, its not going to be faster.
Putting swap on the internal flash will make things slower as the internal flash is about 3 to 5 times slower than a class 6 sdcard.
Click to expand...
Click to collapse
internal memory is faster. class 6 guarantees 6mbs read/write times but doesnt mean the bus can support those times. if you run a test copying something to /system/sd and to /data you will see /data is faster. as for those saying it will degrade the internal memory, that is not the case being the flash memory inside is designed to have much much much more read/write cycles. Think about it in a stock G1, this is where dalvik-cache is writen to as well as email, sms, user settings, cache for browser and uTube. Do not confuse internal flash memory to sd card flash memory
Are you sure about that? This guy has some test results and it indicates that a class 6 sdcard is much faster than the internal flash.
http://forum.xda-developers.com/showpost.php?p=4059520&postcount=15
MonkySlap said:
internal memory is faster. class 6 guarantees 6mbs read/write times but doesnt mean the bus can support those times. if you run a test copying something to /system/sd and to /data you will see /data is faster. as for those saying it will degrade the internal memory, that is not the case being the flash memory inside is designed to have much much much more read/write cycles. Think about it in a stock G1, this is where dalvik-cache is writen to as well as email, sms, user settings, cache for browser and uTube. Do not confuse internal flash memory to sd card flash memory
Click to expand...
Click to collapse
dwang said:
Are you sure about that? This guy has some test results and it indicates that a class 6 sdcard is much faster than the internal flash.
http://forum.xda-developers.com/showpost.php?p=4059520&postcount=15
Click to expand...
Click to collapse
That's interesting. Going to have to try that test out. I just noticed when xfering stuff to /data it was faster then to /system/sd. If it is then touche my friend.....touche
Questions on external storage - SD cards - seem to go on and on and on...
There is a large amount of confusion about them and the information (and mis-information) that exists is all over the web.
Which one is the fastest? The best? How do I partition and format them? Or, the best for my device?
There is no simple answer since the performance of these cards depends on many factors, not of the least of these being the card itself.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Note that I do NOT have a DONATE button anywhere.
I am not looking for donations.
If you feel that you should donate something, by all means,
send it to your favorite XDA developer and/or XDA itself!
And don't be shy about the
button for the many posters who were of help to you!
If you don't like to read you are likely in the wrong thread.. Anyway, if you just want my bottom line - here it is..
Purchase and use a quality card - they will often perform better.
Generally, though not always, higher speed rating means faster.
Learn how to optimise your device (read buffers, caches, etc. all impact performance)
Unofficial recommendation: SanDisk has been a good performer and reliable in my experience.
Find a SanDisk card compatible with your device at the SanDisk Product Compatibility page (thanks Szczepanik!).
Counterfeit card info in post 2.
Formatting info in post 3.
COMMONLY USED CARD TYPES
The most commonly used card type for mobile phones is the SD (Secure Digital) card. It includes 4 families available in different form factors. The four families are the original Standard-Capacity (SDSC), the High-Capacity (SDHC), the eXtended-Capacity (SDXC), and the SDIO, which combines input/output functions with data storage. The three form factors are the original size, the "mini" size, and the "micro" size. There are many combinations of form factors and device families.
DEFINING BASIC CARD SPEED
SD memory card manufacturers use different types of flash memory to create cards, so actual transfer speeds can vary. Varying speeds make it difficult to determine which card will provide reliable recording of streaming content (which is what most manufacturers seem to be concerned with these days).
The two primary classifications are Speed Class and UHS Speed Class.
"Speed Class" has been used by many as a guide to their performance. The Class of a card is a general indication of the read/write speed. Unfortunately things are not that simple in real life. A card's speed depends on many factors, a few of which are:
The likelihood of soft errors that the card's controller must re-try. Writing data requires the controller to read and erase a larger region, then rewrite that entire region with the desired part changed.
Buffers used by the operating system which, in combination with the card's format, sector size and even the device's card bus speed can impact performance.
The possibility of fragmentation - that a body of information the host views as a unit is written to non-contiguous regions of memory. This does not cause rotational or head-movement delays as with magnetic media, but it does vary the amount of computation the card's controller must do.
Speed Classes 2, 4, and 6 assert that the card supports the respective number of MB/s as a minimum sustained write speed for a card in a fragmented state. Class 10 asserts that the card supports 10 MB/s as a minimum non-fragmented sequential write speed.
The following speed classes are defined:
Class 2 - 2 MB/sec
Class 4- 4 MB/sec
Class 6- 6 MB/sec
Class 10 - 10 MB/sec
UHS-I (UHS class 1) - a theoretical 50 MB/sec
USH-II - a theoretical 312 MB/sec
The Ultra-High Speed (UHS) designation is available on some SDHC and SDXC cards. UHS rated cards support a clock frequency of 100 MHz (a quadrupling of the original "Default Speed"), which in four-bit transfer mode could transfer 50 MB/s or a clock frequency of 208 MHz, which could transfer 104 MB/s. Double data rate operation at 50 MHz (DDR50) is mandatory for microSDHC and microSDXC cards labeled as UHS-I. In this mode, four bits are transferred when the clock signal rises and another four bits when it falls, transferring an entire byte on each full clock cycle. UHS-II cards further raise the data transfer rate to a theoretical maximum of 312 MB/s.
It should be noted that it is quite possible for a good Class 4 card to outperform a not so good Class 10 card in various devices (including the HTC HD2 which is what I am using at the time of this writing) as a result of various factors, including card quality, buffers, read/write errors, operating system and application software methodology used.
So, in theory Class 10 is faster than Class 4. As an example, recording video requires a constant minimum write speed while random access (such as our phones mostly use) does not rely on the same type of access. This is really great if you are often recording long videos but I suspect most of us use our phones a little differently.
Often you can get higher speeds from a card than it is rated for. For example, my Class 4 32 GB SanDisk card reads much faster than one might expect it and all too often it is the read speed that makes the device "snappy" (like loading an app).
RANDOM TESTS
It should be noted that these tests were not done by myself. There are many places where tests are described and many methodologies used for them. Benchmarking card performance is not a simple task.
Some of the cards tested include Sandisk 16G class 4, Memorystar 8 G class 10, Memorystar 16 G Class 10, Kingston 8 G class 10, Kingston 16 G class 10, Lexar HS Mobile 32 G class 10, Lexar HS Mobile 16 G class 10, Patriot LX 16 G Class 10, Adata 8 G class 6, Samsung plus 8 G class 6, Sandisk 32G class 4.
Random reads of 512 KB blocks show the SanDisk Mobile Ultra microSDHC to be the fastest card at 21.8 MB/s.
Reducing the block size for the random read test to 4 KB significantly impacts performance, which tops out at a mere 3.4 MB/s, achieved by the Samsung and Adata Class 6 cards. There is almost no difference between queue depths of one and 32 when it comes to memory cards.
Click to expand...
Click to collapse
Click to expand...
Click to collapse
You can review the entire test and the results here.
The random tests and results, which impact mobile phones more, are located here.
BENCHMARKING YOUR CARD
Benchmarks are a rather elusive science since there are too many variables that can impact the results, not the least of which is the card itself. Anyone who has worked with SD cards in the real world has probably noticed that the brand of card can make a dramatic difference in performance. Tests have shown as much as a 12x difference in performance between best and worst cards in real world tasks. As already noted, the "class" ratings used by manufacturers as a measure of performance are not reliable indicators of performance either. In fact, some manufacturer's class 2 and class 4 rated cards routinely outperform cards rated as class 6 or class 10!
Consistency of testing is important since different devices and bottlenecks, such as bus speeds, buffers, file systems and activity can all impact and alter the results. This is why a card that has performed extremely well on a test may turn out to be a dog in your device. Short of testing all cards in a lab environment under exacting circumstances may well yield inconsistent results.
There are a lot of apps available for benchmarking, for Android, as well as Windows and Linux. The following short list is not a recommendation of any software.
HD Tune is a hard disk utility with many functions. It can be used to measure the drive's performance, scan for errors, check the health status (S.M.A.R.T.), securely erase all data and much more. It was designed for hard disks and it has not been updated in recent times. While it could be used for SD Cards (and has been) it is not the ideal tool for it.
More info is available here.
AndroBench is a popular and up to date benchmark application that measures the storage performance of Android devices. It includes Micro benchmarks, SQLite benchmarks and Macro-benchmarks (see screenshot below).
More info is available here.
Crystal Disk Mark is another hard disk test utility which is also somewhat dated (last revision dates back to 2010 as far as I know). It runs under various versions of Windows and Internet Explorer.
More info is available here.
AnTuTu Benchmark is Android Benchmarking tool for Android devices. It can run a full test, through the "Memory Performance","CPU Integer Performance","CPU Floating point Performance","2D 3D Graphics Performance","SD card reading/writing speed","Database IO" performance", etc. The many tests it can perform and the frequent updates make it a viable tool for benchmarking.
More info is available here.
Vellamo is a reasonably easy-to-use suite of system-level benchmarks for devices based on Android 2.3 and later. Some of the functions it offers include CPU subsystem performance, scrolling and zooming, 3D graphics, video performance, memory read/write and peak bandwidth performance.
[ EDIT ] The app has recently been updated and is available on Google Play and is now compatible with KitKat.
More info is available here.
SD Tools is probably one of the most often used tools to check microSD cards (Name, Date, MID, OEMID). You can check if your card is fake. (Check serial number and MID and OEMID). You can also benchmark sd card writing and reading speeds. It is fast and easy to use. See screenshot below.
More info is available here.
Here's my 32GB Sandisk Class 4 card on the Android speed test:
And here is the same card as seen through Androbench:
FILE SYSTEMS FOR SD CARDS
The traditional view has been that storage is not a huge factor for performance on mobile devices. Flash storage (the type most commonly used today) draws little power, and its performance is thought to exceed that of the network subsystem. Yet, oddly enough, just by varying the flash storage, performance over WiFi can typically vary between 100% to 300% across applications; in one extreme scenario the variation jumped to over 2000%. The relationship between storage and application performance seems to be a combination of flash device performance, random I/O from application databases, and use of synchronous writes. Changes to the storage subsystem can significantly improve user experience.
The three primary (not sole) factors impacting device performance as it relates to storage seem to be the media (the card itself), the file system used and the quality of the applications used.
Speed Class is largely irrelevant as it is not necessarily indicative of application performance; although the class rating is meant for sequential performance, there are several cases in which higher-class SD cards performed worse than lower-class ones overall. If not addressed, lower performing storage not only makes the application run slower, it also increases the energy consumption of the device. This part is intended to deal with file systems used in mobile devices and will not address media or application quality (sorry, no tutorial on proper programming techniques) in great detail although they both have a significant impact on performance.
Briefly, Android (as it pertains to storage) consists of flash storage, operating system and Java middleware, and applications; the OS itself is based on Linux and contains low-level drivers (e.g., flash memory, network, and power management), Dalvik virtual machine for application isolation and memory management, several libraries (e.g., SQLite, libc), and an application framework for development of new applications using system services and hardware.
The Dalvik VM is a fast register-based VM providing a small memory footprint; each application runs as its own process, with its own instance of the Dalvik VM. Android also supports true multitasking and several applications usually run as background processes; processes continue running in the background when you leave an application (e.g., a browser downloading web pages).
Android uses SQLite database as the primary means for storage of structured data. SQLite is a transactional database engine that is lightweight, occupying a small amount of storage and memory; it is thus popular on embedded and mobile operating systems. Applications are provided a well defined interface to create, query, and manage their databases; one or more SQLite databases are stored per application on the data partition.
The YAFFS2 file system, managing raw NAND flash was traditionally the file system of choice for the various internal partitions including /system and /data; it is lightweight and optimized for flash storage. Recently, Android transitioned to Ext4 as the default file system for these partitions. Several other files systems have been implemented with varying success in Android devices.
More than 50 file systems have been documented for Linux alone and attempting to document all of them here is simply not possible for me. So many file systems, such little time! You can find plenty of information on all them by doing only a few internet searches. I will concentrate on relatively few of them here. Note that not all attributes of a given file system may be implemented in a given Android system. The omission of a file system from this post should not reflect negatively on it, but rather on my time, or lack thereof.
YAFFS (Yet Another Flash File System) was designed and written by Charles Manning. It is a log-structured file system that holds data integrity as a high priority. Yaffs1 works on NAND chips that have 512 byte pages + 16 byte spare areas. Newer NAND flash chips have larger pages (2048 bytes + 64 bytes spare areas) and stricter write requirements. Each page within an erase block (128 kilobytes) must be written to in sequential order, and each page must be written only once. YAFFS2 was designed to accommodate these newer chips.
YAFFS has been used on Linux, WinCE, pSOS, eCos, ThreadX, and various special-purpose OSes. YAFFS initialization simply erases flash memory. When a bad block is encountered, it follows the smart media scheme of marking the fifth byte of the block's spare area. Blocks marked as such remain unallocated from then on. To write file data, YAFFS initially writes a whole page (chunk in YAFFS terminology) that describes the file metadata, such as timestamps, name, path, etc. The new file is assigned a unique object ID number; every data chunk within the file will contain this unique object ID within the spare area. YAFFS maintains a tree structure in RAM memory of the physical location of these chunks. When a chunk is no longer valid (the file is deleted, or parts of the file are overwritten), YAFFS marks a particular byte in the spare area of the chunk as ‘dirty’. When an entire block (32 pages) is marked as dirty, YAFFS can erase the block and reclaim the space. When the filesystem's free space is low, YAFFS consolidates a group of good pages onto a new block. YAFFS then reclaims the space used by dirty pages within each of the original blocks.
When a YAFFS system mounts a NAND flash device, it must visit each block to check for valid data by scanning its spare area. With this information it then reconstitutes the memory-resident tree data structure.
The extended file system, or ext, was implemented in 1992 as the first file system created specifically for the Linux kernel. It has metadata structure inspired by the traditional Unix File System and was designed by Rémy Card. It was the first implementation that used the virtual file system and it could handle file systems up to 2 gigabytes in size.
The ext2, ext3 and ext4 file systems were all derived from this one. Most ext discussions center around ext3 and ext4 in the Android world.
ext3 is a journaled file system that is commonly used by the Linux kernel. Its main advantage over ext2 is journaling, which improves reliability and eliminates the need to check the file system after an unclean shutdown. Generally, ext3 is slower than competing Linux filesystems, such as ext4, JFS, ReiserFS and XFS, but it has a significant advantage in that it allows in-place upgrades from ext2 without having to back up and restore data. Benchmarks suggest that ext3 also uses less CPU power than ReiserFS and XFS. It is also considered safer than the other Linux file systems, due to its relative simplicity and wider testing base. ext3 does not do checksumming when writing to the journal and if the hardware is doing out-of-order write caching, you run the risk of severe filesystem corruption during a crash.
ext4 was created as a series of backward compatible extensions to ext3. In January 2010, Google announced that it would upgrade its storage infrastructure from ext2 to ext4. In December 2010, they also announced they would use ext4, instead of YAFFS, on Android. The ext4 advantages include large file system support, extents, persistent pre-allocation and journal checksumming.
NILFS (New Implementation of a Log-structured File System) is a log-structured file system for Linux. It is being developed by Nippon Telegraph and Telephone Corporation (NTT) CyberSpace Laboratories. It uses a copy-on-write technique known as "nothing in life is free", NILFS records all data in a continuous log-like format that is only appended to, never overwritten, a design intended to reduce seek times, as well as minimize the kind of data loss that occurs after a crash with conventional file systems. For example, data loss occurs on ext3 file systems when the system crashes during a write operation. When the system reboots, the journal notes that the write did not complete, and any partial data writes are lost. NILFS also includes fast write and recovery times, minimal damage to file data and system consistency on hardware failure, 32-bit checksums, etc.
Android kernels do not routinely include NILFS although mods to make it available can be found.
F2FS (Flash-Friendly File System) was created by Kim Jaegeuk at Samsung for the Linux operating system kernel. The motivation for it was to build a file system that from the start takes into account the characteristics of NAND flash memory-based storage devices, which have been widely used in computer systems ranging from mobile devices to servers. Samsung chose a log-structured file system approach, which it adapted to newer forms of storage. F2FS also remedies some known issues of the older log structured file systems, such as the snowball effect of wandering trees and high cleaning overhead. Because a NAND-based storage device shows different characteristics according to its internal geometry or flash memory management scheme (such as the Flash Translation Layer), Samsung also added various parameters not only for configuring on-disk layout, but also for selecting allocation and cleaning algorithms. Introduced in the second half of 2012 this new file system shows promise but is not yet generally available in any Kernels that I have seen. Samsung has submitted these patches for integration into the Linux kernel, which means it’s likely to appear on Android releases in the future.
JFFS2Journalling Flash File System version 2 or JFFS2 is a log-structured file system for use with flash memory devices. It is the successor to JFFS. JFFS2 has been included in the Linux kernel since the 2.4.10 (2001-09-23) release. JFFS2 is also available for a few bootloaders, like Das U-Boot, Open Firmware, the eCos RTOS and the RedBoot. Most prominently JFFS2 is used in OpenWrt. At least three file systems have been developed as JFFS2 replacements: LogFS, UBIFS, and YAFFS.
JFFS2 adds support for NAND flash devices which have a sequential I/O interface and cannot be memory-mapped for reading. It does not include hard links (this was not possible in JFFS because of limitations in the on-disk format) but it does have compression. Four algorithms are available: zlib, rubin, rtime, and lzo. It claims better performance - JFFS treats the disk as a purely circular log which generats a great deal of unnecessary I/O. The garbage collection algorithm in JFFS2 makes this mostly unnecessary.
Today's predominant file system, YAFFS2, will likely be replaced in the future by the likes of ext4, nilfs or f2fs. In order to do a fair comparison one must compare I/O throughput, user data access latency, application execution latency and data safety. Not a simple task.
If you ever want to get Linux techies arguing just talk about which file systems are the best.
Google, which knows a thing or two about fast systems, has decided, for their purposes anyway, that Ext4 is the best and close to the fastest file system of all. Google also hired Ted T'so, who also happens to be the leading Ext4 programmer. In a note to the Ext4 developer mailing list, Google's Michael Rubin, a senior staff engineer, wrote, "Google is currently in the middle of upgrading from ext2 to a more up to date file system. We ended up choosing ext4." So, if you are using an Android phone and you are not a kernel developer you may want to take Google's word for it, at least for now, and go with the ext4 file system on your SD Card.
In all fairness, the numerous tests that have been ran over the years will prove different winners. Some show ext2 to slightly outperform ext4 but we must also consider data safety and journaling. While not many of us will have vital, enterprise data on our mobile devices, reconfiguring and restoring a device can be tedious, at best. Some will argue for NILFS, others for ext4, and yet others.. well, you get the idea.
I cannot tell you which is the best or fastest or safest. What I can tell you is that, based on my experience, I am staying mostly with ext4 for the time being - reasonable speed and safety combination for my needs.
Comments? Additions? Suggestions? They are all welcome.
Flame wars (relating to SD Cards or otherwise) are not. :-]
HOW TO identify counterfeit Secure Digital Cards -and- backup your card to your PC!
IDENTIFYING COUNTERFEIT CARDS
A significant number of buyers have been deceived by inferior quality, cheap, slow SD cards rebadged as SanDisk or other brand names for quick profit. Many buyers get scammed by fraudulent sellers and products each day. There are a number of blogs showing images of fake cards.
This post is intended to help those who already have the card and would like to be certain.
With flash memory being able to be manipulated into displaying a set or upgraded fake capacity, there must be a way to efficiently test the flash memory. Not only is this testing critical for end users, it is essential for product manufacturers further up the supply chain to have a reliable way to detect fake Nand flash, otherwise they will be unwittingly producing products with incorrect capacities and creating marketplace chaos as well as soiling their reputation. The industry standard for testing memory is the burn in test, which essentially writes a set amount of data onto the memory, and then verifies said data. Errors signify that the memory is unstable and of lower quality (downgrade), or possibly that the memory has been upgraded to a fake capacity.
By far the most widely used and long-standing champion of burn-in testing for Nand flash is the h2testw.exe program, or affectionately known as the H2 burn-in test. Other benchmark and burn in testing programs have come and gone, all defeated by the upgraders in China. H2 was originally written by Harald Bogenholz for c't Magazin (Magazin für Computertechnik), a German computer magazine, and has been used extensively in the flash memory industry from China, Taiwan, to Korea, since 2008. The same version 1.4 has been in use since 2008 and has never been updated, which just goes to prove the reliability of Mr Bogenholz's awesome burn-in testing program.
Here's how:
Insert the flash memory card that will be tested into a reader and launch the H2 program.
Using the Select Target button, choose the drive letter corresponding with the flash memory device.
Once selected, leave "Data volume: all available space" selected and "endless verify" unselected.
Click the "Write + Verify" button to begin the testing.
H2 will "burn-in" the full capacity of data into the flash memory device, and then verify the burned-in data.
If there are errors, chances are the flash is faulty or has been upgraded to a fake capacity.
As long as the flash IC has been attached onto a PCB with a controller, it is possible the memory has already been faked. Each time flash memory exchanges hands post SMT production, from half finished PCBA, to finished product QC, or the end distributor's inspection; H2 is there every step of the way to verify the memory capacity. Hopefully the champ can continue to defeat all of the most sophistaced hackers and upgraders to ensure everyone gets the memory capacity that they paid for.
BACKING UP YOUR SD CARDS
For those of us who have only Windows systems backing up the ext4 partition with the rest of the card has been a bit of a chore. Not any longer! Thanks to MarkAtHome for finding W32ImageWriter we can now create a complete image backup of the card - all partitions - and restore it in tact! The following is a step-by-step of how I did it.
Win32DiskImager is a freeware project currently in beta release v0.6. It will create and restore an image copy of your entire card to and from your Windows PC without a Linux system. Note that image copy means a sector by sector copy of the card into an image file. The program does not care what is on the card!
You can read more about it here (ignore the Rasberry Pi headings!) - I am not going to duplicate their entire page here, sorry.
Once you download the binary (link below) you can unzip it into any folder of your choice. It does not require "installation". The unzipped content of the download should look like this:
Place your SD Card into a USB card reader - it should have a Windows drive letter assigned to it. Now simply launch W32DiskImager.exe on your PC. The card I was using in my phone was a SanDisk Class 4 32 GB card. Looking at it in Minitool showed the following information. You can see my FAT32 and ext 4 partitions:
Keep in mind that this is beta - it still has some minor issues. Upon launch I received an error which may have been caused by Windows Explorer still being open. It seems this app expects exclusive control of the card although clicking OK continued operation without any further issues.
Once at the main screen of the app there is a "read" and "write" button choice. Select a folder and a file name on your PC for the image you are about to create and click "read" to copy the content of your card to your PC. The program prefers a file extension of .img.
This process will take a while since the entire card is being copied sector by sector, not only the files on the card - be patient. When the process completes you will have a large image file of your entire SD card - roughly the size of the card (again, not the content). To test the process I used a second SanDisk Class 4 32 GB card. It was clean formatted with FAT32 without any additional partitions.
I launched W32ImageWriter again, located the image file I created above, made sure the correct drive letter was shown for my card and clicked "write" to copy the saved image from the PC to the card. This process took considerably longer than the creation process since write speeds to the cards are slower than read speeds. Leave it alone, go get some coffee or tea and relax. If you watch the progress bar you will slow the process. When the restoration completed I looked at the card in Minitool and compared it to the first card - I could not see a difference.
The final test was to put the second card into my phone and power up. All was fine, as before. Just to be certain I didn't mix things up and shut the phone down and inserted the other (the original, I think) card and powered up again. As far as the phone was concerned the cards were identical.
And, I now had a complete card image saved on my PC, including the FAT32 and ext4 partitions, which I could recreate any time on the same or another card! Very cool!
A few caveats! As noted above W32ImageWriter is still in beta. There is a known memory leak in this version that has been fixed and will be in the next release. Due to this I recommend running this app in a non-repetitive manner (i.e. use once then exit and restart if necessary).
If you'd like to give it a try you can find the free binary here.
I would appreciate hearing about your experience with it if you try it.
Formatting and Troubleshooting
PARTITIONING AND FORMATTING
The many file systems available for these cards is beyond the scope of this post but a brief mention is needed, nonetheless. Likewise, deciding on read buffers and cluster sizes depend a great deal on your operating system, how the card is used, the speed of the card bus and other factors. You will have to read and experiment more if you want to get optimum performance from your card.
Because the host views the SD card as a block storage device, the card does not require MBR partitions or any specific file system. The card can be reformatted to use any file system the operating system supports such as UFS, Ext2, Ext3, Ext4, btrfs, HFS Plus, HFS Plus, NTFS, FAT16, FAT32, exFAT, etc.
Most consumer products that take an SD card will expect it to be partitioned and formatted in some way. The universal support for FAT16 and FAT32 allow the usage of SDSC and SDHC cards on most host devices. On such SD cards, standard utility programs can be used to repair a corrupted filing system and sometimes recover deleted files. Defragmentation tools for FAT file systems may be used on such cards but are generally not recommended. The resulting consolidation of files may provide a marginal improvement in the time required to read or write the file, but not an improvement comparable to defragmentation of hard drives, where storing a file in multiple fragments may involve a time penalty to move between physical areas of the drive. Moreover, defragmentation performs writes to the SD card that count against the card's rated lifespan.
An SD card should have a life span of roughly 10 years (which doesn't mean it always will). In theory, the more often you write/erase it the faster it will wear. Of course, replacing an SD card is fairly inexpensive these days.
The memory of a card is divided into minimum memory units. The device writes data onto memory units where no data is already stored. As available memory becomes divided into smaller units through normal use, this leads to an increase in non-linear, or fragmented storage. The amount of fragmentation can reduce write speeds, so faster SD memory card speed standards help compensate for fragmentation.
Reformatting an SD card with a different file system, or even with the same one, may make the card slower, or shorten its lifespan. Some cards use wear leveling, in which frequently modified blocks are mapped to different portions of memory at different times, and some wear-leveling algorithms are designed for the access patterns typical of the file allocation table on a FAT16 or FAT32 device. In addition, the preformatted file system may use a cluster size that matches the erase region of the physical memory on the card; reformatting may change the cluster size and make writes less efficient.
The SD Association provides free formatting software to overcome many problems described above. This software formats all SD memory cards, SDHC memory cards and SDXC memory cards. It was created specifically for memory cards using the SD/SDHC/SDXC standards. It is generally recommended to use the SD Formatter first if the card is to be reformatted by another method. Using generic formatting utilities may result in less than optimal performance for your memory cards.
Nota bene: cards greater than 32GB will be automatically formatted as exFat by this software! If you require another file system, such as FAT32, you will have to do a second format with another utility! If you need to format a 64 GB card to FAT32 you can use various utilities to get it done, including this FAT32Format program.
The SD/SDHC/SDXC memory cards have a "Protected Area" on the card for the SD standard's security function. The SD Formatter does not format the "Protected Area". This is genrally reserved for the use of the device, such as cameras and mobile phones.
To create other partitions, such as an "ext4" Linux formatted partition, grab a copy of MiniTool Partition Wizard. The Home Edition is a free partition manager software designed by MiniTool Solution Ltd. It supports 32/64 bit Windows Operating Systems. Functions include: resizing partitions, copying partitions, create partition, extend partition, split partition, delete partition, format partition, convert partition, explore partition, etc.
Similar to the MiniTool Partition Wizard is the Paragon Partition Manager which also has a free version you can download. This one will allow you to increase an ext4 partition's cluster size, if you wish to.
This is a very good article on partition alignment if you are interested in knowing more about that subject - thanks MarkAtHome!
The free SD Association formatting software is available here.
The free Minitool Partition Manager software is available here.
The free Paragon Partition Manager software is available here.
There are many ways you can format a card, including various recovery tools, some bootloaders, PC card readers, etc. I am not suggesting that other methods do not work or that they are bad. This is simply my preferred way of getting it done.
Let's take a 32 GB SanDisk card and set it up for some of the devices. I am using a card reader attached to my PC for these steps. If this is not a brand new card the first step I will take is format it with SD Formatter to make sure it is in optimal form. Loading the program on your PC will give you a screen like this one:
This program will create the "proper" format for your card. Note that this means a 64 GB card which was originally sold with the eXFat format will be so formatted. If your device requires FAT32 format (which many phones require) you will have to do that with another program.
Note the "Option" button! Clicking this button will let you select the format type (quick, full or erase). I generally select full although it takes quite a bit longer. I am seldom concerned with securing old data from other people. I do, however, like to have format adjustment turned on.
When SD Formatter is finished you should see something like this:
If you need only a FAT32 partition you can stop here. You are done. If your setup requires a Linux type partition also for apps or data or whatnot you keep going.
My ROM uses the extended partition for most everything except the Android system. The advantage of this separation is that reads and writes will deal with two different devices, hopefully making things a little faster than waiting for just one.
I prefer to use the ext4 file format which includes journaling (see writeup on file system for more info). I will accomplish the creating of this partition with the Minitool Partition Manager. When you first load the program you should see something like this image although the number of drives and their order will depend on your system.
Note that the FAT32 partition is marked as "Primary"! This was done automatically by SD Formatter but if you are using another program you must remember to do this yourself or chances are the partition will not been properly seen on your device!
Creating the second, ext4 type partition also requires it to be set as "Primary"!
WARNING!! This program requires you click the apply button in order for your selections to be completed. If you do everything and forget to click apply nothing will actually be done!!
As you can see, the FAT32 partition spans the entire card. The simplest way to create room for another partition is to first decrease the size of the FAT32 partition. You can select the "Partition" menu choice on top or right click the FAT32 partition and you will see a "Move/Resize" choice. Selecting it will bring you to this window:
Reduce the size of the FAT32 partition and you will see available space increase as you are doing it. I am aiming for a 3 GB ext4 partition:
Once you apply the changes you should see the unallocated space available for the second partition:
Right click the newly created space and select "Create" partition. Don't forget to set the new partition as "Primary"! Since I am not planning to create another partition (such as Linux swap) I am using all the free space on the card.
Select Apply again and you should see your second partition being created:
Once the process is completed both of your partitions should be clearly shown and, if you did it right, both will be flagged as "Primary" partitions! Now, that wasn't so difficult, was it?
A FEW WORDS ABOUT PARTITION TYPES
There is apparently a lot of information floating around on the topic of "partition types". While they are fairly well defined and documented I could find nothing specific about requiring FAT32 type "b" partitions for Android. That does not mean I could not have missed the info but one would think it should be quite easy to find if this was a blanket requirement.
I read through Tytung's git repositories (okay, kinda speed reading since there is way too much source code there for limited time reading) and I could not find any restrictions on FAT32 type, though again, admittedly I could have missed it.
As importantly, I created a FAT32 partition as type "b" on one card and another as type "c" on another card - they both worked equally well for me. Minitool Partition Wizard creates type "c" (the FAT32 default for the program) unless you change it yourself. I have apparently never used anything other than type "c" on my HD2, on Typhoon GB ROMs, and Tytung's ICS and JB ROMs. So, you got me. If it matters I have not seen that yet.
Briefly, the partition type (or partition ID) in a partition's entry in the partition table inside a Master Boot Record (MBR) is a byte value intended to specify the file system the partition contains and/or to flag special access methods used to access these partitions, such as CHS (Cylinder/Head/Sector) mappings, LBA (Logical Block Addressing) access, logical mapped geometries, special driver access, hidden partitions, secured or encrypted filesystems, etc.
It is up to an operating system's boot loader and/or kernel how to interpret the value.
Type "b" was the original (or older) WIN95 OSR2 FAT32 design. It is limited to partition sizes of 2047GB or less and it depended on the BIOS INT 13 of those PCs. Type "c" also dates back to WIN95 OSR2 FAT32 but is LBA-mapped and is considered an extended-INT 13 equivalent of 0b. Of course, both of these specs were designed for actual (hard) disk drives. According to some notes I have seen the "c" revision should be faster but there is no concrete evidence of this as far as I have seen.
By changing the partition type ID, users can prevent the system from using or initializing partitions - this is up to the operating system which will be accessing it.
According to Microsoft the following limitations exist using the FAT32 file system with Windows operating systems:
Clusters cannot be 64 kilobytes (KB) or larger. If clusters were 64 KB or larger, some programs (such as Setup programs) might calculate disk space incorrectly.
A volume must contain at least 65,527 clusters to use the FAT32 file system. You cannot increase the cluster size on a volume using the FAT32 file system so that it ends up with less than 65,527 clusters.
The maximum possible number of clusters on a volume using the FAT32 file system is 268,435,445. With a maximum of 32 KB per cluster with space for the file allocation table (FAT), this equates to a maximum disk size of approximately 8 terabytes (TB).
The ScanDisk tool included with Microsoft Windows 95 and Microsoft Windows 98 is a 16-bit program. Such programs have a single memory block maximum allocation size of 16 MB less 64 KB. Therefore, The Windows 95 or Windows 98 ScanDisk tool cannot process volumes using the FAT32 file system that have a FAT larger than 16 MB less 64 KB in size. A FAT entry on a volume using the FAT32 file system uses 4 bytes, so ScanDisk cannot process the FAT on a volume using the FAT32 file system that defines more than 4,177,920 clusters (including the two reserved clusters). Including the FATs themselves, this works out, at the maximum of 32 KB per cluster, to a volume size of 127.53 gigabytes (GB).
You cannot decrease the cluster size on a volume using the FAT32 file system so that the FAT ends up larger than 16 MB less 64 KB in size.
You cannot format a volume larger than 32 GB in size using the FAT32 file system in Windows 2000. The Windows 2000 FastFAT driver can mount and support volumes larger than 32 GB that use the FAT32 file system (subject to the other limits), but you cannot create one using the Format tool. This behavior is by design. If you need to create a volume larger than 32 GB, use the NTFS file system instead.
Keep in mind the above is Windows (Microsoft). Linux and Android handle things a little differently. All of the Linux filesystem drivers support all three FAT types, namely FAT12, FAT16 and FAT32. As far as I know Android should handle them the same way. Other common features that they all support are various Linux mounting options (specified with the -o option to the mount command), such as "uid", "gid", "umask", etc.
There really is so much more to partitioning and formatting that I simply cannot cover it all here, however, all the information is already available someplace on the internet. Search is a wonderful tool, everyone!
SOME OTHER SD CARD RELATED STUFF TO PONDER
Finding and Fixing (?) Bad Blocks
SD cards include controller circuitry to perform bad block management and wear leveling. Although there is software to find bad blocks considering the price of these cards today if you're starting to see bad blocks the device is just plain going bad. Time to get a new one before you have serious problems.
You could run diagnostics to try and find/fix it if you really want to. This can be done on your PC and a card reader or in Android Terminal:
Code:
su
fsck_msdos /dev/block/yourdevice
For ext2/3/4 partitions you can use e2fsck but it needs to run on an unmounted partition. If this is not clear you probably should not attempt it until you read a lot more, sorry.
You cannot fix bad blocks. Once a block is bad - it remains bad. Kind of like trying to fix a battery that cannot hold a charge any longer.
There are many supposed fixes claiming to do magic. None actually work. You may be able to tell the system a block is bad and it should not be used, thus avoid problems, but there is no fix.
Eventually, when your device develops too many bad blocks (around 50% of NAND) it will go to join that big phone company in the sky - time to get a new phone.
The best way to check bad blocks is to format all partitions in ClockWorkMod (warning: this will wipe your phone completely!) and check the recovery log when finished - after the format go to Advanced in ClockWorkMod Recovery and choose report error, this will save your log in your /sdcard/clockworkmod You can view this file with most text editors or copy/email it to your computer for viewing.
Alternately, you can try to reboot your device. When the boot process is completed (before you do anything else) load the TERMINAL program and enter the following:
Code:
su
dmesg > /sdcard/dmesg.txt
This will create a text file called dmesg.txt in the root of your card which can be viewed with a text editor. Look for the kernel name early in the file. Bad blocks, if any, should be listed not too far after the name. Or try Lumberjack.
Thanks to Robbie P for the reminder on a HD2/MAGLDR option:
Robbie P said:
If using MAGLDR, go to services/DMESG to SD to get Dmesg from last boot.
Click to expand...
Click to collapse
Since we seem to be talking about bad blocks - it has been some time since this info was originally posted. For everyone who missed it (or forgot), here it is again.
The HD2 aka Leo is manufactured with 2 different NAND Flash ROM chipsets.
You can find the type by entering the tri-color boot mode.
If you see PB81120 SS-B3 on the first line that means your NAND Flash ROM is from Samsung (KBY00U00VM)
If you see PB1120 HX-B3 on the first line your NAND Flash ROM is from Hynix (H8BFS0WU0MCR)
After about one year of usage the Samsung chipset usually has a dozen or so bad clusters.
Hynix normally has none of them or just a couple.
A quick way to delete the Dalvik cache from your ext4 partition.
There are many, many ways to get this done should it be desired. One quick and easy way to do it in Android Terminal is:
Code:
su
cd /data/dalvik-cache
rm *
exit
Reboot your device and wait for the boot process to rebuild the cache.
Checking the system logs for problems
Problems with SD Cards as well as NAND and applications are often recorded in one of the Android system logs. Developers are often asking for logs so they can identify the source of a complaint. While not difficult to get, these logs are not in plain view. One the quick and easy ways to view and save the Android logs is a utility called Lumberjack.
The free Lumberjack software is available here.
If you prefer to use Android Terminal you can dump a lot of this info into text files which you can view later or email. The following will dump DMESG, KMSG and LOGCAT to your SD Card:
Code:
dmesg > /sdcard/dmesg.txt
cat /proc/kmsg > /sdcard/kmesg.txt
cat /dev/log/system > /sdcard/logcat.txt
Comments? Additions? Suggestions? They are all welcome.
Flame wars (relating to SD Cards or otherwise) are not. :-]
Thanks. Very interesting information.
Sent from my Nexus 7
Benchmark
ny_limited said:
Place holder .......
Click to expand...
Click to collapse
Great thanks it would be helpful to have guidance on the various Android apps that benchmark read and write speed, for example
AnTuTu
Androbench
Cheers
Tom
Szczepanik said:
Great thanks it would be helpful to have guidance on the various Android apps that benchmark read and write speed, for example
AnTuTu
Androbench
Cheers
Tom
Click to expand...
Click to collapse
Good idea, thanks! Not enough time at the moment but I'll see if I can come up with some..
i'm using Transcend 16GB Micro-SDHC (Class10)
and using swapper2, SD Card Boost
lucubrb said:
i'm using Transcend 16GB Micro-SDHC (Class10)
and using swapper2, SD Card Boost
Click to expand...
Click to collapse
About swappers....
Swapper2 is an app that lets you run a virtual memory on your sd card instead of using the internal phone memory. All swappers will decrease the life of your card (or NAND) simply by their nature, but SD cards are pretty cheap these days.
The intent of swapping is to create "make believe" RAM for working processes by moving applications in and out of working memory.
Unlike traditional swap, Android's Memory Manager kills inactive processes to free up memory. Android signals to the process, then the process will usually write out a small bit of specific information about its state (for example, Google Maps may write out the map view coordinates; the browser might write the URL of the page being viewed) and then the process exits. When you next access that application, it is restarted: the application is loaded from storage, and retrieves the state information that it saved when it last closed. In some applications, this makes it seem as if the application never closed at all. This is not much different from traditional swap, except that Android apps are specially programmed to write out very specific information, making Android's Memory Manager more efficient than swap.
Personally, I much prefer tweaking the Memory Manager settings than using swap files.
About SD Card Boost...
Not sure what this is. Is this something like the read buffer or something else?
Szczepanik said:
Great thanks it would be helpful to have guidance on the various Android apps that benchmark read and write speed, for example
AnTuTu
Androbench
Cheers
Tom
Click to expand...
Click to collapse
Updated post 1 with a few benchmark apps in no particular order or recommendation. Some are dated while others quite popular. I suspect that the methodology used for benchmarking - keeping all variables the same - is possibly more important than the test app itself.
Wow... this is what I have been looking for, hence it has made it to my signature :good:
Ultimate guide about SD cards.
apurohit said:
Wow... this is what I have been looking for, hence it has made it to my signature :good:
Click to expand...
Click to collapse
Thanks, but it is a work in progress..
I have a few more ideas to put up (and another thread or so) but time is not always cooperating.
So sad when when work gets in the way of having fun!
New ROM Toolbox
The very versatile ROM Toolbox was just updated and Jared has essentially re-written some SD Card related code. It may just be worth checking out. There isa a Lite and Full version - sorry I do not recall which features are paid only.
ROM Toolbox is the must have app for every root user. ROM Toolbox combines all the great root apps into one monster app with a beautiful and easy to use interface. ROM Toolbox has every tool you need to make your Android device fast and customized to your liking.
The October 23, 2012 update adds a re-written SD Booster (you can now set read buffers as high as you wish!) and new SD benchmarks & storage chart.
More info is available here.
ny_limited said:
About swappers....
Swapper2 is an app that lets you run a virtual memory on your sd card instead of using the internal phone memory. All swappers will decrease the life of your card (or NAND) simply by their nature, but SD cards are pretty cheap these days.
The intent of swapping is to create "make believe" RAM for working processes by moving applications in and out of working memory.
Unlike traditional swap, Android's Memory Manager kills inactive processes to free up memory. Android signals to the process, then the process will usually write out a small bit of specific information about its state (for example, Google Maps may write out the map view coordinates; the browser might write the URL of the page being viewed) and then the process exits. When you next access that application, it is restarted: the application is loaded from storage, and retrieves the state information that it saved when it last closed. In some applications, this makes it seem as if the application never closed at all. This is not much different from traditional swap, except that Android apps are specially programmed to write out very specific information, making Android's Memory Manager more efficient than swap.
Personally, I much prefer tweaking the Memory Manager settings than using swap files.
About SD Card Boost...
Not sure what this is. Is this something like the read buffer or something else?
Click to expand...
Click to collapse
what buffer size good for card?
coohdeh said:
what buffer size good for card?
Click to expand...
Click to collapse
Interesting question. Too often people tend to make very large or very small read buffers and performance suffers. Ideally, the buffer should parallel the cluster size of your card.
The best cluster size can usually be determined by using SD Formatter. 32k clusters are not always the best although they generally are for big cards (e.g. 32 GB and bigger).
Using 1024KB or 2048KB buffer with 32 KB clusters is, in my view, rather inefficient since it will save your device several read operations to read multiple clusters (which is the card's unit, so to speak).
In my experience setting a read buffer which is not less than 100% and not more than 500% of the cluster size is ideal. Experimenting is generally easy and a good idea as your 'mileage may vary'. Please remember to experiment - while a large buffer may do wonders (as it did for me) it is not always the best choice. See this post for a little contrast.
To set the read buffer the init.d scripts used to do the job - they still work, of course. The latest version of ROM Toolbox lets you set the buffer to any size you want and it will automatically reset it on each boot (a necessary evil). This is a very easy way to experiment without rebooting after each change.
Currently, my cluster size is 32KB and my read buffer is 16MB (16384 KB).
Using 1024KB and 2048KB read buffer my devise was extremely sluggish!
File systems for SD Cards - the debate goes on!
I have updated post 3 of this thread: File systems for SD Cards - the debate goes on!.
Have fun!
great
Wow. Now that was a lot of infos in the morning. I never thought that I ever gonna find so detailed and article about SDcards. Thanks
crancker said:
Wow. Now that was a lot of infos in the morning. I never thought that I ever gonna find so detailed and article about SDcards. Thanks
Click to expand...
Click to collapse
You mean you read it all?! I am impressed!
Was my typing / spelling OK?
I am using samsung 32GB class 10. It's really fast in my Samsung!
ny_limited said:
I have updated post 3 of this thread: File systems for SD Cards - the debate goes on!. Have fun!
Click to expand...
Click to collapse
ny_limited thanks again. It will be good to move any debate about file systems out of Xylograph's thread.
Meanwhile I recommend that readers of this thread view it in a browser and, in the top right corner, rate it five star and "newsworthy".
Cheers
Tom
Sent from my HTC HD2 using Tapatalk
Are there any reasons to use a class 10 memory card on a Android phone?
Essentially, most of the apps are installed on your internal memory of the phone itself. Isn't the memory card just used as a storage medium for music and videos?