So.. You've plugged your SD to your linux box and want to created aligned partitions with fdisk?
Follow exactly what I'm doing and your partitions will be MB aligned.
At the end you'll get a FAT32 partition for your data and an ext4 partition for data2xxxx scripts.
If you need more partitions, just adapt this tutorial to you needs.
I know some scripts or recoveries do this automatically so no need to remind me
I initially posted this in SuperNova ROM thread but it might worth a separate thread.
I've done this with an old 1GB usb key just to show you the commands, you'll have to adapt to your SD depending on it's size.
When I plugged the flash drive, it got recognized by my system as /dev/sde. It can be different on your system and once again you'll have to figure-out yourself the correct device.
Here as an example I'm splitting the flash drive in 2 partitions. 1st (512Mb) will be for the fat32 partition and will have to be formatted later. The rest is the future ext4 and is about 512MB too.
I'm gonna use 2 magic parameters with fdisk, in order to get 1024Kb clusters (2048x512 bytes) and align easily the partitions. 64 heads and 32 sectors should give 64x32=2048 so that is going to be my params here
Code:
fdisk -H64 -S32 /dev/sde
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sde1 * 1 1005 1028043+ 6 FAT16
Partition 1 has different physical/logical beginnings (non-Linux?):
phys=(0, 3, 45) logical=(0, 7, 10)
Partition 1 has different physical/logical endings:
phys=(509, 63, 63) logical=(1004, 3, 32)
Partition 1 does not end on cylinder boundary.
So here I have an old partition on my key (sde1). I delete it first and check that there is no more partition on the drive.
Repeat the process if you have more than 1 partition on your SD.
Make sure that it's clean before proceeding to the next step.
Code:
Command (m for help): d
Selected partition 1
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
Empty! Great! The key is clean. Let's partition it now
Code:
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1005, default 1): 2
Last cylinder or +size or +sizeM or +sizeK (2-1005, default 1005): 512
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sde1 2 512 523264 83 Linux
Ok now I have a partition starting on cylinder 2 and 512Mb in size. If your SD is 16GB, you can use for instance a size of 15GB to leave 1GB for the ext4 partition.
That would give approximately 15x1024=15360 instead of 512. Got it?
Pay attention to the value below "End". Here it is 512 so I will use the next cylinder (513) for the 2nd partition:
Code:
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (1-1005, default 1): 513
Last cylinder or +size or +sizeM or +sizeK (513-1005, default 1005):
Using default value 1005
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sde1 2 512 523264 83 Linux
/dev/sde2 513 1005 504832 83 Linux
Now I have 2 partitions aligned to 1MB but declared as "linux". Let's change that to FAT32 for partition 1.
Code:
Command (m for help): t
Partition number (1-4): 1
Hex code (type L to list codes): c
Changed system type of partition 1 to c (W95 FAT32 (LBA))
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sde1 2 512 523264 c W95 FAT32 (LBA)
/dev/sde2 513 1005 504832 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks
That's all.
Now format 1st partition to FAT32 with this guide:
credits to kd8cgo
Code:
Making an aligned FAT32 format
Anyone can figure out alignment of the partition from many sources already posted on the web.
One thing I could not find was a guide on how to align that pesky FAT32 format so that each
cluster (allocation unit) is aligned inside the boundaries of erase blocks. Well, the Linux mkfs.vfat
utility has all the options we need to make this aligned format possible!
Please view this primer to FAT32 to get an idea of how the system is laid out on your disk.
You normally have the first sector in the partition which is your Volume ID, which sits in the
reserved space which is normally the first 32 sectors in the partition. This is followed
by (2) copies of the File Allocation Tables, which vary in length when created depending on
a variety of factors, including the chosen allocation unit size. The FAT size remains constant
after creation. Please keep in mind for this discussion the physical size of a sector, which is 512 bytes.
The Golden Nugget
We will be changing the reserved sectors from the default value of 32, to a number that
we will calculate from the reported size of the FAT tables after formatting. The goal will be to
make the FAT tables end right at a 128K boundary, so each cluster of the file system will fall
neatly within erase blocks on our disk!
First we will format our disk using FAT32 paying no mind to reserved sectors. This will
report to us our FAT size as so:
[email protected]:~$ sudo mkfs.vfat -F 32 -n UserData -s 32 -v /dev/sde1
mkfs.vfat 3.0.7 (24 Dec 2009)
/dev/sdd1 has 224 heads and 56 sectors per track,
logical sector size is 512,
using 0xf8 media descriptor, with 31309312 sectors;
file system has 2 32-bit FATs and 32 sectors per cluster.
FAT size is 7641 sectors, and provides 977937 clusters.
Volume ID is 40c250bd, volume label UserData .
The juicy bits are that 2nd to last line, it tells you the size on disk of 1 FAT table.
7641 sector FAT x 2 FATs x 512 bytes/sector = 7,824,384 bytes
The above formula shows you the exact amount of space the FAT tables are using at
the beginning of your disk. This number is not usually going to be evenly divisible by
128K (131,072 bytes) as you can see 7,824,384 / 131072 = 59.695 erase block sized
chunks. What we need to do is force the end of those FAT tables to end right at
60 blocks to do so we:
131,072 x 60 = 7,864,320 bytes in 60 erase blocks
7,864,320 - 7,824,384 = 39,936 bytes remainder
39,936 / 512 = 78 sectors remain
New reserved sector count for alignment = 78
Those are all the fundamentals required to align a FAT32 partition, so that
clusters on the disk fall in line with the erase blocks of the physical cell medium.
An example of the format command required:
sudo mkfs.vfat -F 32 -R 78 -n UserData -s 32 -v /dev/sde1
Breakdown:
sudo - super user privledges
mkfs.vfat - create a FAT file system
-F 32 - 32 bit FAT (FAT32)
-R 78 - Use 78 Reserved Sectors (instead of 32)
-n UserData - drive label, up to 11 characters
-s 32 - 32 sectors per cluster 32 x 512 - 16K allocation unit size
-v /dev/sdex - Device to format
Note: If you're getting a R=0, use 256 (256x512bytes=128Kb) as the minimum R value for FAT32 is 2.
Format 2nd partition with the following command, replacing xxx with your device
Code:
mke2fs -T ext4 -b 4096 -E stride=64,stripe-width=64,hash_alg=tea -O extent,^huge_file -m 0 -L userdata /dev/xxx2
You should be good to go now!
Reserved for later
thanks for this you absolute babe <3
Did you read post #7 by CitizenJohn a few posts below the original HOWTO: Increase write speed by 'aligning' FAT32
Tends to contradict the method posted by kd8cgo for Fat32
Presumably both methods can't be correct!
Of course I did. I read many posts about SSDs to come to this tutorial.
First of all I don't use the same geometry for creating partitions.
And so far this way of formatting has always been able to give me the best of all my flash memory based devices, SD and USB keys.
Have you tried both methods? Which of the two gave you the best results?
I'm gonna try it again, maybe I missed something the first time.
Of course if there is a proven better way to get even more from our flash drives and SDs, I'll gladly learn it and change the tutorial accordingly.
EDIT:
Posted atto results for both methods on a pretty old flash drive. Left is formatted with the post you mentioned and right is with the "standard" method.
Don't see much of a difference.
I have but quite a while apart, i didn't use any software to measure transfer speed but using a sd-card adapter according to Linux i get ~10Mbs sd-card to PC and ~8Mbs PC to sd-card with the method i mentioned can't imagine a class2 sd-card could get much faster
Damn just realized to PC was Fat partition to PC : To sd-card was to btrfs linux partition
The problem is that we don't know the actual erase block size of each flash drive.
And on top of that, fat32 has block sizes of 64K max. Max erase block size for SD is 256K but is usually less.
I also just tried both methods on a faster usb key (with write speeds up to 15Mb/s) and results are the same.
But each his own and if anybody can get a better data transfer rate with the 2nd method, please report all the details here.
No worries mate thanks for the TUT, the question came out of ignorance rather than any understanding of the methodology
ziggy1001 said:
No worries mate thanks for the TUT, the question came out of ignorance rather than any understanding of the methodology
Click to expand...
Click to collapse
Actually it was an interesting question.
We assume that for any device the erase block size is 128K
But the guy in your post is formatting also a usb flash thumb drive or an SD card.
If the erase block size of his flash drive is 16k, then his file was aligned
details:
8011776 / 131072 = 61.125 File "not aligned" according to him.
8011776 / 16384 = 489. Round number, file aligned.
To get even more speed, one has to use bigger fat32 blocks (up to 64k). But then each file will use blocks of 64Ko on the drive, even if it's real size is 1Ko !
Good compromises are 16 and 32Ko blocs.
... Or you can use GParted.
problem solved.
Terepin said:
... Or you can use GParted.
problem solved.
Click to expand...
Click to collapse
Could be a good idea but:
1/ Some people have reported issues with Gparted when aligning their partitions and even some 16GB SDs were not detected by the Desire after using Gparted. On the opposites, the phone could detect them by doing it manually.
2/ Unless I'm mistaken, Gparted is NOT optimizing the FAT32 formatting process nor the EXT4 one.
is there really a difference between Gparted live and your method ?
when you wipe your SDext with your recovery, does sdext returns to default value ?
t-minik said:
is there really a difference between Gparted live and your method ?
when you wipe your SDext with your recovery, sdext returns to default value ?
Click to expand...
Click to collapse
Yes there is: I don't think that Gparted is using 1024Ko cylinders.
But most of the time, choosing MB alignment is working to align partitions. However, it is not aligning the filesystem. And like I already said, some people reported issues, that's why I published this tutorial.
If Gparted is working for you, keep using it
yep, GP works ..... but I'm really curious
and what happens when wiping SDEXT ?
If you change -s to 128 does that give 64k block size?
sudo mkfs.vfat -F 32 -R 78 -n UserData-s 32 -v /dev/sde1
Well whatever it was transfer Fat32 to PC went from ~10Mbs to ~16Mbs
If you change -s to 128 does that give 64k block size?
sudo mkfs.vfat -F 32 -R 78 -n UserData-s 32 -v /dev/sde1
Well whatever it was transfer Fat32 to PC went from ~10Mbs to ~16Mbs
Click to expand...
Click to collapse
yep, s128 gives 64k blocks.
Did you calculate R78 with the given method?
Happy that you're getting higher speed, but usually this method goal is to increase write speed to the SD
Sent from my A70S using the XDA mobile application powered by Tapatalk
Does 4ext recovery does this aligned partitioning/formatting?
Great thread, Sibere.. Will modify my installation steps to point to this TUT than Sd formatter
Does 4ext recovery does this aligned partitioning/formatting?
Click to expand...
Click to collapse
According to the dev, it should. Didn't try myself though. but like Gparted, it won't optimze the FAT32 formatting part.
Sent from my A70S using the XDA mobile application powered by Tapatalk
LiooO said:
Does 4ext recovery does this aligned partitioning/formatting?
Click to expand...
Click to collapse
sibere said:
According to the dev, it should. Didn't try myself though. but like Gparted, it won't optimze the FAT32 formatting part.
Sent from my A70S using the XDA mobile application powered by Tapatalk
Click to expand...
Click to collapse
So according to your answer i can partiton SD card in 4Ext recovery or GParted and then just do Fat32 formatting according to your guide? Can someone confirm this pls?
Hello,
First I'd like to tell that I've already searched the forum using Google for my problem, but I'm really new to Android and almost all posts are using lots of acronyms that I don't understand. Eventually I found myself with so many tabs opened in my browser that I almost lost what I was looking for at first.
I have a rooted (with Unrevoked) HTC Desire (Clockworkmod Recovery is in place), on which I managed to install Cyangenmod 7.0.3 a few months ago. I can't recall how I get to this result unfortunately. At the time, I only remember that I fiddled a lot because although there are plenty of howtos for many different things, they look all different and/or specific to one device, and the newbie user is not warned about the potential differences he could face.
Now I would like to upgrade to Cyanogenmod 7.1 (for bug fixes) and install DarkTremor App2SD (because I've created an ext4 partition long ago on my SD-card, naively thinkg it would be used, but it stays desperately empty; and the phone internal memory is almost full despite my strenuous attempts to move as much apps as possible to the SD card using, as far as I can tell, CyanogenMod's Apps2SD).
I stumbled on a nice video explaining "how to upgrade Cyanogenmod 7" (search this exact string on Youtube): basically, it uses ROM Manager to install the new Cyanogenmod without wiping (this is probably something that the ROM supports between minor versions). As I understand it, I should be able to use this method to install DarkTremor App2SD as well.
So I've download the Cyanogenmod 7.1 zip file and even copied in as "update.zip" in the root of my FAT32 partition of my SD card. Unfortunately, Clockwordmod Recovery spits the following message:
Code:
Waiting for SD Card to mount (12s)
E: Can't mount /dev/block/mmcblk0 (or /dev/block/mmcblk0p1)
(Invalid Argument)
Waiting for SD Card to mount (11s)
E: Can't mount /dev/block/mmcblk0 (or /dev/block/mmcblk0p1)
(Invalid Argument)
[...]
Timed out waiting for SD Card...
and then goes back to the recovery menu.
My SD card is partitioned with the following scheme (output is from Linux parted on /dev/sdb, which is accessed through an USB cable with USB storage activated on the phone):
Code:
(parted) p
Model: HTC Desire (scsi)
Disk /dev/sdb: 16.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 16.4kB 67.1MB 67.1MB primary
2 67.1MB 537MB 470MB primary ext4
3 537MB 16.0GB 15.5GB primary fat32 lba
As you can see, the first partition is a swap partition (not sure it has been ever used though), the second one in my ext4 partition where I want to move my apps and the third one in the FAT32 partition which contains update.zip.
Thus my understanding it that /dev/block/mmcblk0(p1) points at best to the swap partition. Am I correct?
I could use another temporary SD card to perform the update, but this is unwieldy and given this partitioning scheme looks pretty common among the Android community, I wonder there's an easier way and/or if I missed something.
Please, avoid "me too" replies . And you would be nice if you avoided using acronyms as much as possible when replying (or at least, give a definition upon first use).
Thanks a lot for your help.
Cheers,
-- Tatazou
tatazou said:
Hello,
My SD card is partitioned with the following scheme (output is from Linux parted on /dev/sdb, which is accessed through an USB cable with USB storage activated on the phone):
Code:
(parted) p
Model: HTC Desire (scsi)
Disk /dev/sdb: 16.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 16.4kB 67.1MB 67.1MB primary
2 67.1MB 537MB 470MB primary ext4
3 537MB 16.0GB 15.5GB primary fat32 lba
As you can see, the first partition is a swap partition (not sure it has been ever used though), the second one in my ext4 partition where I want to move my apps and the third one in the FAT32 partition which contains update.zip.
Thus my understanding it that /dev/block/mmcblk0(p1) points at best to the swap partition. Am I correct?
-- Tatazou
Click to expand...
Click to collapse
I think I got this more or less confirmed: I got into the phone with "adb shell" and checked /proc/partitions:
Code:
major minor #blocks name
31 0 640 mtdblock0
31 1 5120 mtdblock1
31 2 2560 mtdblock2
31 3 256000 mtdblock3
31 4 40960 mtdblock4
31 5 151168 mtdblock5
179 0 15637504 mmcblk0
179 1 65520 mmcblk0p1
179 2 458752 mmcblk0p2
179 3 15113216 mmcblk0p3
Apart from the internal storage, you can see the mmcblk0 device and its partitions whose sizes match the parted output above.
I just bought a new Quality micro SD from Samsung, 32GB EVO. I was using a brandless 32GB.
I used Win32DiskImager program (windows) to create images for both SD cards. One has everything installed what I want.
The newer Samsung has only the basic apps like link2SD, rootinternal2external.
The strange part is that there is an difference in the amount of free storage, both are parttioned with the same program.
The only difference is that the amount of partitions:
Black brandless: 3 partitions: Data,App,swap
Samsung: Data , App
But the image file that I pulled from the SD cards is as follows:
Black: 31.691.112.448 MB
Samsung: 31.440.502.784 MB
How come there is an difference?
I want to burn the old image on my new card. I guess I can make a partition of the black SD card about 260MB smaller, then create
a new image again, then burn it. I hope that works.
But is it normal that the amount of storage differs between brands???
I think that yes, regards
[How to] Copy image between different size micro SD
I found a way to copy my larger SD image, made with Win32diskimager, into my smaller Samsung 32GB micro SD:
http://softwarebakery.com/shrinking-images-on-linux
Basicly you have to do:
- Use a tool to access the micro SD and make the last partition XXX MB smaller. This creates unpartitioned space.
- Create a image file from that micro SD: xxxx.img
- Boot into a Linux Operating system, Like Ubuntu 14.04LTS, by installing it on your HD or boot directly from a live CD
- Open a terminal screen. With CTRL+ALT+T, or use the graphical UI to open one terminal.
- in that terminal, execute the command: fdisk -l xxxx.img
- The output of the command looks like the following:
Disk myimage.img: 6144 MB, 6144000000 bytes, 12000000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000ea37d
Device Boot Start End Blocks Id System
myimage.img1 2048 9181183 4589568 b W95 FAT32
Now we have to calculate where we want to cut the xxxx.img file. We want to cut away the last part that is not allocated to any partition.
Above you see the line "myimage.img1 2048 9181183 4589568 b W95 FAT32".
The last partition ends at 9181183, in your example it will be different ofcourse, just take the end of the last partition.
If you have 2 partitions, then you have 2 lines under:
"Device Boot Start End Blocks Id System"
in this example we have to cut from 9181183+1. Identify also what the block size is. In our example it's 512.
512 is the block size. In your example it might be different. If it's different, execute below command with a different value.
Now we cut away the empty space that is not used in our xxx.img.
Execute in the terminal screen: truncate --size=$[(9181183+1)*512] xxxx.img
The xxxx.img file is now smaller and can be burned into the smaller device.
I want to know that when we flash specific partition images (say system.img, data.img, cache.img, etc.), then how does the software know where exactly to put those partition bytes? For example, if I flash system.img, which CHS/LBA sector will it consider as beginning of that block? If it is based on the MBR/EBR1 tables, what will happen if I'm flashing the MBR/EBR1 too?
The reason I'm asking is that I want to change the partitioning in EBR1 slightly, so that more space is allocated to /data partition instead of /sdcard. I've got an old but good conditioned MediaTek (MTK-6577) based smart-phone called Karbonn-A30 which is great in build quality and almost every other aspect, but it only has 500MB of Internal Storage (/data partition in Linux lingo) which is not good enough for apps. Presently, the MBR and EBR1 partitions are thus:
$disktype MBR
--- MBR
Regular file, size 512 bytes
DOS/MBR partition map
Partition 1: 2.000 TiB (2199023255040 bytes, 4294967295 sectors from 1024)
Type 0x05 (Extended)
Partition 2: 10 MiB (10485760 bytes, 20480 sectors from 18432)
Type 0x83 (Linux)
Partition 3: 10 MiB (10485760 bytes, 20480 sectors from 38912)
Type 0x83 (Linux)
Partition 4: 650 MiB (681574400 bytes, 1331200 sectors from 113152)
Type 0x83 (Linux)
$disktype EBR1
--- EBR1
Regular file, size 512 bytes
DOS/MBR partition map
Partition 1: 376 MiB (394264576 bytes, 770048 sectors from 1443328)
Type 0x83 (Linux)
Partition 2: 1.293 GiB (1388314624 bytes, 2711552 sectors from 2213376)
Type 0x83 (Linux)
Partition 3: 1.998 TiB (2196501691904 bytes, 4290042367 sectors from 4924928)
Type 0x83 (Linux)
Click to expand...
Click to collapse
Why this last partition which corresponds to /sdcard is left so large (1.998 TiB) is beyond my understanding! Since there is a good 2.5GB of space available on my actual /sdcard partition, I was thinking if I can alter the EBS1 and change the LBA addressing so that the third partition starts from 2097152 additional sectors (which comes to 1024MB or 1GB which is good enough for me), will it automatically increase my /data partition by 1GB and decrease the /sdcard by 1GB correspondingly? On XDA and other forums, I've read that people have successfully done this mod and achieved the change in partition sizes, but I first want to understand how it happens.
I've made this mistake before, and it drives me nuts. I changed phones and put my old phones sd card in my new phone (Nokia 8.3). I was just going to see if it could read it. It couldn't and asked me if I wanted to format. I selected no, and removed the card. I put the card in my linux computer and I can't mount it. I did get a replacement for my old phone through warranty (teracube) and tried putting it in that, and it wouldn't read it either. I've had this problem before. I think even having the sdcard in a phone does something to it even if you don't format it. I think I formatted the card initially as ext2 but I could be mistaken.
Any help would be appreciated. I really want to recover the data.
mount:
Code:
[3630 21-01-23 7:06][email protected]/home/larry%sudo mount /dev/sda2 /mnt/tmp
NTFS signature is missing.
Failed to mount '/dev/sda2': Invalid argument
The device '/dev/sda2' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
[3630 21-01-23 7:06][email protected]/home/larry%sudo mount -tvfat /dev/sda2 /mnt/tmp
mount: /mnt/tmp: wrong fs type, bad option, bad superblock on /dev/sda2, missing codepage or helper program, or other error.
fdisk:
Code:
[3630 21-01-23 7:06][email protected]/home/larry%sudo fdisk /dev/sda
Command (m for help): p
Disk /dev/sda: 119.1 GiB, 127865454592 bytes, 249737216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: C8FBA780-1A1F-4B71-8D9D-D623A671C61D
Device Start End Sectors Size Type
/dev/sda1 2048 34815 32768 16M unknown
/dev/sda2 34816 249737182 249702367 119.1G unknown
An SD-card ( from Android's point of view it's a portable USB-device ) usually is FAT32 formatted, AFAIK, but I may err as always ...