Database Info

Free space gone - du / df difference on /data?

I'm having a strange problem with my jf-1.43-adp1.1holiday.
The free space on my /data is 28 MB, according to df (after moving apps, but not data to SD). The thing is, if I run du on my /data, it says only 16MB is occupied.
# df
/dev: 49520K total, 0K used, 49520K available (block size 4096)
/sqlite_stmt_journals: 4096K total, 0K used, 4096K available (block size 4096)
/system: 69120K total, 67172K used, 1948K available (block size 4096)
/system/modules: 1532K total, 1532K used, 0K available (block size 4096)
/system/xbin: 3172K total, 3172K used, 0K available (block size 4096)
/system/sd: 1475568K total, 1002632K used, 472936K available (block size 4096)
/data: 76544K total, 47920K used, 28624K available (block size 4096)
/cache: 69120K total, 4528K used, 64592K available (block size 4096)
/sdcard: 14044744K total, 12176704K used, 1868040K available (block size 8192)
# cd /data
# du -h -x -d 1 -a
512 ./app
2.0K ./tombstones
369.5K ./system
63.5K ./anr
3.5K ./property
2.0K ./app-private
6.9M ./data
6.0K ./local
36.0K ./misc
8.7M ./dalvik-cache
2.0K ./lost+found
16.1M .
So, (48MB - 16MB) space is just missing.
The constant "low free space" message was the reason I moved the apps to SD, but turns out the apps didn't take much space, something else did.
All my caches are cleared.
Anyone know what's going on?
Thanks

Did you move the various cache's to the sd card?
I know the marketplace one grows pretty quickly.

mitch1974 said:
Did you move the various cache's to the sd card?
I know the marketplace one grows pretty quickly.
Click to expand...
Click to collapse
I cleared all my caches where the program gives me an option to do so.
AFAIK, market cache is cleared automatically if there's not enough free space. Since I had a permanent "space is getting low" notification before I moved the apps to SD yesterday, I think the market cache is somewhat empty. Besides, it would show up in du output.

yaffs2 on android sometimes leaks inode counts (well, sort of). Try this:
The proper way to work around this it to shut down your phone by
holding down the end call button until you get a dialog asking to shut
down your phone. Turn your phone back on. BEFORE your phone can sleep
after it reboot, shut it down again. This will clean your unlinked
files.
Click to expand...
Click to collapse
Full info:
http://www.mail-archive.com/[email protected]/msg19510.html

aziwoqpd said:
yaffs2 on android sometimes leaks inode counts (well, sort of).
Full info:
http://www.mail-archive.com/[email protected]/msg19510.html
Click to expand...
Click to collapse
Thanks for the link!
I tried it, rebooted about 6 times just in case, didn't help. The df/du difference is exactly the same as it was before.
Maybe a wipe will help?
By the way, is there any difference between JF's wipe option during boot and adp -w?
Here's my /proc/yaffs output after the last reboot, if this helps:
# cat /proc/yaffs
YAFFS built:Jan 24 2009 21:00:58
$Id$
$Id$
Device 0 "system"
startBlock......... 0
endBlock........... 539
totalBytesPerChunk. 2048
nDataBytesPerChunk. 2048
chunkGroupBits..... 0
chunkGroupSize..... 1
nErasedBlocks...... 10
nReservedBlocks.... 5
blocksInCheckpoint. 1
nTnodesCreated..... 2800
nFreeTnodes........ 95
nObjectsCreated.... 500
nFreeObjects....... 26
nFreeChunks........ 1482
nPageWrites........ 0
nPageReads......... 46893
nBlockErasures..... 0
nGCCopies.......... 0
garbageCollections. 0
passiveGCs......... 0
nRetriedWrites..... 0
nShortOpCaches..... 10
nRetireBlocks...... 0
eccFixed........... 0
eccUnfixed......... 0
tagsEccFixed....... 0
tagsEccUnfixed..... 0
cacheHits.......... 0
nDeletedFiles...... 4
nUnlinkedFiles..... 10
nBackgroudDeletions 0
useNANDECC......... 1
isYaffs2........... 1
inbandTags......... 0
Device 1 "userdata"
startBlock......... 0
endBlock........... 597
totalBytesPerChunk. 2048
nDataBytesPerChunk. 2048
chunkGroupBits..... 0
chunkGroupSize..... 1
nErasedBlocks...... 11
nReservedBlocks.... 5
blocksInCheckpoint. 0
nTnodesCreated..... 3000
nFreeTnodes........ 24
nObjectsCreated.... 1900
nFreeObjects....... 93
nFreeChunks........ 13908
nPageWrites........ 988
nPageReads......... 5897
nBlockErasures..... 15
nGCCopies.......... 342
garbageCollections. 15
passiveGCs......... 0
nRetriedWrites..... 0
nShortOpCaches..... 10
nRetireBlocks...... 0
eccFixed........... 0
eccUnfixed......... 0
tagsEccFixed....... 0
tagsEccUnfixed..... 0
cacheHits.......... 263
nDeletedFiles...... 1023
nUnlinkedFiles..... 90704
nBackgroudDeletions 0
useNANDECC......... 1
isYaffs2........... 1
inbandTags......... 0
Device 2 "cache"
startBlock......... 0
endBlock........... 539
totalBytesPerChunk. 2048
nDataBytesPerChunk. 2048
chunkGroupBits..... 0
chunkGroupSize..... 1
nErasedBlocks...... 502
nReservedBlocks.... 5
blocksInCheckpoint. 0
nTnodesCreated..... 200
nFreeTnodes........ 55
nObjectsCreated.... 100
nFreeObjects....... 73
nFreeChunks........ 32854
nPageWrites........ 6
nPageReads......... 17
nBlockErasures..... 0
nGCCopies.......... 0
garbageCollections. 0
passiveGCs......... 0
nRetriedWrites..... 0
nShortOpCaches..... 10
nRetireBlocks...... 0
eccFixed........... 0
eccUnfixed......... 0
tagsEccFixed....... 0
tagsEccUnfixed..... 0
cacheHits.......... 0
nDeletedFiles...... 19
nUnlinkedFiles..... 430
nBackgroudDeletions 0
useNANDECC......... 1
isYaffs2........... 1
inbandTags......... 0
Click to expand...
Click to collapse

How much space does g2x have?

It says the phone has 8 gigs of internal memory but in settings storage is only 1.48 gigs and additional storage 5.44 gigs. It internal memory is only 1.48 gigs how do they come up with the number of 8 gigs on the website. Im yet to see any settings that tell the phone to install everything to that additional storage area. After flashing eagleblood v1.3 and all the apps i have restored with titanium backup ive only got 0.99 gb of internal space now.

The ROM uses the last GB for various things.
Moving an app to the "SD Card" will actually move it to that 5GB~ of storage.

Ok thanks i will start sending some apps over to that location haha. On another note do you know of any hdmi issues with that rom? ive had issues with the picture cutting out for about 2 seconds and then showing again it does it like every 10 seconds while im playing something thru hdmi out.

After reading of all the problems people had with HDMI cables shorting out the phone, I never bothered to buy an HDMI cable for it, myself!

Wow really? shorting the phone out? i would have a dam tantrum if that happened while watching something with the hdmi. No shorting for me just constant black screen cutout while video continues to play on the phone

Lightning Blue said:
The ROM uses the last GB for various things.
Click to expand...
Click to collapse
Interesting. Could you explain that further or provide a link? Curious to see what exactly is happening.

jcbofkc said:
Interesting. Could you explain that further or provide a link? Curious to see what exactly is happening.
Click to expand...
Click to collapse
I don't have a link for it but the space for the ROM/OS has to come from somewhere!
Here's an output from "busybox df -h" run from a terminal, minus the app2sd stuff:
Filesystem Size Used Available Use% Mounted on
tmpfs 186.2M 0 186.2M 0% /dev
/dev/block/mmcblk0p6 2.4M 2.0M 412.0K 83% /nvdb
tmpfs 186.2M 0 186.2M 0% /mnt/asec
tmpfs 186.2M 0 186.2M 0% /mnt/extasec
/dev/block/mmcblk0p1 401.9M 315.9M 65.3M 83% /system
/dev/block/mmcblk0p7 1.5G 327.9M 1.1G 22% /data
/dev/block/mmcblk0p2 62.0M 29.8M 32.2M 48% /cache
/dev/block/vold/179:8 5.4G 2.3G 3.1G 43% /mnt/sdcard
Click to expand...
Click to collapse
If you add up the space needed by /system, /dev, data, /cache, /mnt/sdcard ("Internal SD") and the other stuff, you're roughly at 8GB

[Q] /system partition is full - How can I free space?

I just tried to modify /system/framework/framework-res.apk but when pushing it back to the device it says that there is no space left.
output of df is this
Code:
# df
df
Filesystem Size Used Free Blksize
/dev 359M 32K 359M 4096
/mnt/asec 359M 0K 359M 4096
/mnt/obb 359M 0K 359M 4096
/system 251M 251M 2M 4096
/data 28G 3G 25G 2048
/cache 166M 4M 162M 4096
/pds 1M 108K 1M 2048
/mnt/sdcard 28G 3G 25G 2048
I also tried deleting some apps in /system/app to free some space and then /system/framework/framework-res.apk before pushing it back but after deleting it the space became even smaller
Code:
# df
df
Filesystem Size Used Free Blksize
/dev 359M 32K 359M 4096
/mnt/asec 359M 0K 359M 4096
/mnt/obb 359M 0K 359M 4096
/system 251M 251M 56k 4096
/data 28G 3G 25G 2048
/cache 166M 4M 162M 4096
/pds 1M 108K 1M 2048
/mnt/sdcard 28G 3G 25G 2048
Is there anything that could have used the space left on my /system partition?
How can I free more space? Right now my xoom is not working since framework-res.apk is missing
Edit: Tried rebooting and now I'm stuck with the loading moto logo

Have you tried #Xoom on IRC?
You don't have a Hong Kong Xoom, by chance?

[Q] Internal memory

Dear all,
Sensation XE specs state 4 gigs of internal memory and only 1 gig available.
Where is the other 3 gigs?
Could we recover them somehow?
Regards.

padvou said:
Dear all,
Sensation XE specs state 4 gigs of internal memory and only 1 gig available.
Where is the other 3 gigs?
Could we recover them somehow?
Regards.
Click to expand...
Click to collapse
boot partition
data partition
firmware partition
cache partition
system partition
that's why we have only 1gb available for the user

padvou said:
Dear all,
Sensation XE specs state 4 gigs of internal memory and only 1 gig available.
Where is the other 3 gigs?
Could we recover them somehow?
Regards.
Click to expand...
Click to collapse
As rzr86 said, there are a few partitions on the phone, of which only /data is available for us (unless rooted).
System is where the system apps and framework are located (the OS itself)
Boot is where the kernel is.
Firmware is where... the firmware is located
And there are a few more that, like firmware, are blocked unless S-OFF (even if HTC-Unlocked), to keep the device from being bricked (these partitions are responsible for the boot-up and for connecting the software to the hardware.
Other devices with larger internal storage also have this behaviour - models with 16GB usually have 12-13 available to the user. So is 32 and 64 GB versions of phones - about 3-4 GB are reserved for system usage.
As for recovering that area - It can't be "recovered", as it is needed for the OS to function. It's like Windows takes a couple of gigs to be installed (or any other OS).
The unavailable area is all of those partitions.

It is clear to me now.
Thank you for your responses. :good:

It sounds like a big waste of space right? Even if sense uncompressed takes around 1gb something along those partitions that must be a s**tload of space left. Oh well. I guess we have to have fun with the 1gb available for us lol.
PS -AOSP roms should take less than 200mb installed, talk about waste of space :b

MidnightDevil said:
It sounds like a big waste of space right? Even if sense uncompressed takes around 1gb something along those partitions that must be a s**tload of space left. Oh well. I guess we have to have fun with the 1gb available for us lol.
PS -AOSP roms should take less than 200mb installed, talk about waste of space :b
Click to expand...
Click to collapse
Just think about this - with have lots of room to expand our ROMs, we are not limited to just stock. For example, there is an old Optimus One (now unused, being used by me as a testing device for apps) in my house, to run Cyanogenmod, they had to remove all non-English languages, so I had to remove some system apps to put hebrew fonts inside. And there are many more devices with 512MB ROM.
The desire got it's 2.3 update as a "trimmed" version of sense to be able to have it, and on all phones there is almost no place for apps.
Some ROMs don't use all that space, right. But having this overhead benefits us as we aren't limited by space.
Also, in the future, if someone pulls it off - we can have an hboot with the ability to make custom partitions, like the Nexus One has.
I've seen the partition layout just a couple of days ago, will try to find it (incl. sizes).
Edit: Okay, found a shell command to give partition layout, here it is (those are all mounted partitions, not all of them
Filesystem Size Used Free Blksize
/dev 271.7M 52K 271.6M 4096
/storage 271.7M 0K 271.7M 4096
/vendor/firmware/misc 199.8M 21.5M 178.3M 4096
/vendor/firmware/adsp 199.8M 5.7M 194.1M 4096
/mnt/secure 271.7M 0K 271.7M 4096
/mnt/asec 271.7M 0K 271.7M 4096
/mnt/obb 271.7M 0K 271.7M 4096
/mnt/fuse 271.7M 0K 271.7M 4096
/system 787.4M 388.8M 398.6M 4096
/data 1.1G 484.6M 691.4M 4096
/cache 118.1M 98M 20.1M 4096
/devlog 19.7M 14.8M 4.9M 4096
Some of those I know, some are new to me, and they actually seem empty (all those /mnt/ for example), but I believe they have their purpose. If someone not S-OFF (or even not unlocked at all) could check we may have the answers.

How is the internal space managed?

Hi, this is probably a silly question, in the sense that the answer is simple, and yet right now I'm a bit lost. My total internal storage capacity, according to Windows when my phone is plugged, is 226gb. Also according to Windows, and file managers on Android, all the content is taking 180gb. So far so good. But on Windows I see free space: 16gb. So, 226 - 180 = 46.
Removing the 16gb free, that leaves me with something unknown taking 30gb. Is it the system, the apps, the apps data? If it's the system, I think 30gb is way too much. If it's the apps, also I think it's too much. If it's the app's data, that's actually OK, but I thought that data is usually on the internal storage. If not, were could it be?
I think this is a mix of all the three, but, to be certain, I would like to know a way to check the system size, the apps size and the apps data size. Can this be done? Thanks!
EDIT
OK, just found out that system is taking about 14gb and this is already excluded from the total capacity shown on Windows, so, I still don't know where the 30gb occupied space may be coming from.

Check device's internal storage's disk usage via du command. See also here:
How to Find Disk Usage of Files and Directories in Linux
This article explains 10 useful "du" (disk usage) commands that help you to find out disk usage of files and directories in Linux.
www.tecmint.com

Yes, du reports 180GB on internal sdcard, that I know. But if total is 226GB and free is 16GB, 226GB minus 180GB actually give us 46GB supposedly free, so, where are the 30GB left of the occupied space?

AFAIK a portion of the internal sdcard's total capacity is used to store certain functions including optimizations of the memory that support performance and endurance and therefore is not available for user storage.

xXx yYy said:
AFAIK a portion of the internal sdcard's total capacity is used to store certain functions including optimizations of the memory that support performance and endurance and therefore is not available for user storage.
Click to expand...
Click to collapse
It's not available, yet it shows as space taken? Would that take 30GB?

Sorry for making a new post instead of an edit, but there's new info and I think it should be separated. So, I've been playing around with the df and du command, and here are the results. I've deleted a 40GB folder from internal memory so forget about the numbers on the first post:
df output:
Code:
Filesystem Size Used Avail Use% Mounted on
/data/media 226G 160G 67G 71% /mnt/runtime/default/emulated
du output:
Code:
104M /data/adb
36K /data/anr
14K /data/apex
13G /data/app
3.5K /data/app-asec
3.5K /data/app-ephemeral
3.5K /data/app-lib
3.5K /data/app-private
3.5K /data/app-staging
0 /data/app-tmp
1.3M /data/backup
3.5K /data/bootchart
14K /data/cache
906M /data/dalvik-cache
9.2G /data/data
0 /data/data-tmp
3.5K /data/dpm
11K /data/drm
3.5K /data/extm
7.0K /data/gsi
1.4M /data/local
3.5K /data/lost+found
18M /data/magisk_backup_7d97454e4da0aa16d84783dfc5b6e31f1ca44c15
131G /data/media
3.5K /data/mediadrm
25M /data/misc
66K /data/misc_ce
105K /data/misc_de
3.5K /data/miui
2.8M /data/mqsas
11K /data/nfc
3.5K /data/ota
3.5K /data/ota_package
3.5K /data/per_boot
3.5K /data/preloads
12K /data/property
11K /data/rclone
260K /data/resource-cache
40K /data/rollback
3.5K /data/rollback-observer
4.5K /data/ru.bkqovrzm.cvkvlilxi
4.5K /data/ru.guhkdbvd.bioxbkqve
4.5K /data/ru.tsglwgsm.qdvpylesc
3.5K /data/sdcard
7.5K /data/server_configurable_flags
3.5K /data/ss
59M /data/system
4.0M /data/system_ce
289K /data/system_de
3.5K /data/tmp
18K /data/tombstones
27K /data/unencrypted
306M /data/user
805M /data/user_de
168M /data/vendor
18K /data/vendor_ce
7.5M /data/vendor_de
156G total
I have a couple of questions here. Is it the /data/media from df equal to /data folder? If they are, according to df there are 160GB occupied, but according to du theres 156GB in use. Where are the 4GB left?
Also, if those paths does not relate to the same thing, according to df there are 160GB occupied on "/data/media", but according to du theres 131GB in use in "/data/media". Where are the 29GB left?
I hope my questions make sense. I feel like I'm missing something here about how spaces a partitions work on Android, so let's hope someone can clarify me that.
Thanks!

Only to have mentioned it:
Android typically ( ext4 FS ) allocates disk space in 4KB chunks. Means the disk space allocated for files you store always is a multiple of 4KB. So storing a file of size 1B will allocate 4KB, if you store a file of size 5KB then 8KB disk space get allocated ...
Disk space allocated for a file in the rarest case is file's real size.

xXx yYy said:
Only to have mentioned it:
Android typically ( ext4 FS ) allocates disk space in 4KB chunks. Means the disk space allocated for files you store always is a multiple of 4KB. So storing a file of size 1B will allocate 4KB, if you store a file of size 5KB then 8KB disk space get allocated ...
Disk space allocated for a file in the rarest case is file's real size.
Click to expand...
Click to collapse
So you mean those 4GB left may came from that fact? So the "/data/media" from df command is the "same" one as "/data/" from du command?

Data/media is its own thing and sits next to data/data, data/obb.. AFAIK its a new thing and contains all the media related to apps and games etc.. Presuming video and audio files.. I wouldn't delete it as it might bork apps but you might be able to delete certain ones if they won't affect the OS..
Any storage discrepancy will be in cluster size and bits to bytes conversions.

shivadow said:
Data/media is its own thing and sits next to data/data, data/obb.. AFAIK its a new thing and contains all the media related to apps and games etc.. Presuming video and audio files.. I wouldn't delete it as it might bork apps but you might be able to delete certain ones if they won't affect the OS..
Any storage discrepancy will be in cluster size and bits to bytes conversions.
Click to expand...
Click to collapse
I'm talking about the difference in labels between commands.
According to df:
Code:
Filesystem Size Used Avail Use% Mounted on
/data/media 226G 160G 67G 71% /mnt/runtime/default/emulated
According to du:
Code:
131G /data/media
156G /data/ (total)
So, if the "/data/media" label from df equals the "/data/media" label from du, I'm missing 30GB (160GB used vs 131 GB used). If the "/data/media/" label from df it's actually the data partition, then I'm missing 4GB (160GB used vs 156GB used). You get what I mean? If they're the same I could understand 4GB discrepancy, if they are not the same, are you saying the filesystem format actually eats up 30GB out of nowhere???

In your case /data/media is a mounted partition ( block device ) with fixed maximum size 226GB. It houses the directories /storage emulated/<UID>/Downloads, /storage emulated/<UID>/Musics, /storage/emulated/<UID>/Pictures, storage emulated/<UID>/Videos, ...
The mentioned partition may already have so-called bad blocks, i.e. unusable blocks. It is not uncommon for sd-cards to have 2-6% bad blocks when they leave the factory.

xXx yYy said:
In your case /data/media is a mounted partition ( block device ) with fixed maximum size 226GB. It houses the directories /storage emulated/<UID>/Downloads, /storage emulated/<UID>/Musics, /storage/emulated/<UID>/Pictures, storage emulated/<UID>/Videos, ...
The mentioned partition may already have so-called bad blocks, i.e. unusable blocks. It is not uncommon for sd-cards to have 2-6% bad blocks when they leave the factory.
Click to expand...
Click to collapse
And the /data folder is also inside this partition, right? Because that one is taking 156GB according to du... if not, why according to df /data/media has 160GB occupied and according to du /data/media has 131GB occupied? Why this discrepancy?

OMG. Become familiar with your device's Android OS's partition structure. Learn what is difference between partition and directory. GIYF ...

I'm trying, but I haven't found anything solid enough so I'm asking here.