Database Info

Observations as to why market breaks / force close, and other anomolies

As I suspected early on the issues boil down to corruption within the User Data or Cache partitions, less often on the system partition due to an unexpected shutdown of the device. Shut on these devices need to follow the proper shutdown routine as any linux environment. Following this best practice will ensure that all data is written out to its corresponding file system by flushing all cache, unmounting the file system, etc..
Here are the culprits of why we see so frequent random Force Closes, Market Resetting, etc. ultimately resulting in an unclean shutdown, corrupting some data.
1. The button we use is also a forced off button. Typically if you hold it down too long you are powering off the device.
2. Some times when in sleep mode you see the Viewsonic logo upon starting - that means that the system shutdown (most likely crashed).
3. If your running Vegan your hitting the reboot.. I dont know for sure but I suspect this is NOT performing a clean shutdown... (I dont have a copy of the source)
Anyway... wanted to pass this on... as last night my data partition became corrupt after using the Reboot function on the Poweroff menu of Vega 5.1..

shouldnt need source code to debug a dirty shutdown..Cant you just run an adb logcat? maybe run the shutdown command in a terminal on the device and pipe the output into a text file for later viewing

My internal memory has to be repartitioned every few weeks - I'm certain that something is corrupting it over time. I had massive FC's just a week or so back where the SD partition re-do was the only fix.
I suspect that this happens in stock, as well - the problem of course is that there is no fix for a stock user, other than a return / exchange.

roebeet said:
My internal memory has to be repartitioned every few weeks - I'm certain that something is corrupting it over time. I had massive FC's just a week or so back where the SD partition re-do was the only fix.
I suspect that this happens in stock, as well - the problem of course is that there is no fix for a stock user, other than a return / exchange.
Click to expand...
Click to collapse
I have been on stock since I got the device just moving to the newer versions when they come as OTAs and have never ever had to mess with my partition, so I don't THINK the issue is in the stock software. In fact, the only problems I've ever encountered were when I used the enhancement pack, in which case my screen started to become unresponsive and the calibration.ini I was told to try did not work. Since then I went back to 3389 and the device has been perfect ever since.
I could be wrong though and just very, very lucky....here's to hoping. Another thing to consider is maybe the memory is going bonkers for some reason. I've had flash memory that lasted forever and I've had flash memory that has gone wacky over a period of 6 months....even a wipe by the utility designed to do it doesn't fix it properly. I don't know how CWM wipes or partitions the memory, I do know there's supposed to be a special way to do it.
If it's not faulty memory off the bat, then that leaves something in the 'extras' being put into these ROMs. Maybe some of the newer tegra drivers or some coding to make the ROMs faster - I'm just saying, can't leave any stone unturned.
Has anyone that has stayed loyal to stock encountered these issues? We have to ask that question I think. Then we ask how many of the people playing with ROMs are seeing the issues, this would include people that have used CWM to partition and mess with their mounts initially.
I can say I've never seen data disappear from my internal memory or my SD and I can also say I've never seen multiple FCs except after putting in the enh. pack (keep in mind I got my tab on Dec 20something, so I had 3053 and then 3389 soon after).
The first sign of anything being 'corrupted' on it's own at stock and I'll be sending mine back. As an owner of Android since Android's been around, I've never had my G1 or MT4G (or any smartphone before it) become corrupted due to not being shutdown or reboot properly and while this is a tablet I think the fundamentals should be the same. Pampering 'faulty' memory is a risk. You can wipe and re-do all you want, but if it's faulty it's going to stay that way.

Ive done that but I guess you can say unfortunately I have had only clean shutdowns since then... The last corruption I had I formatted my data and cache partitions before I ran logcat.... Of course thought of that afterward....
Generally if any has FCs, etc. etc. run a logcat and post it here... we will be able to confirm this...
We could change the way the partitions are created and add a sync which will further reduce chances BUT will take a performance hit...
I am very surprised though as the EXT3 filesystem is very resilient to dirty shutdowns (more than EXT4)...
I reviewed the out of the box framework source on the google GIT and technically if a reboot command is given a clean shutdown is performed via the framework... but the widget on the shutdown screen I suspect is not calling the method properly or is not being called at all... All speculation at this point... But for sure there is corruption occurring..
Since the last corruption I switch over to pershoots kernel... Even though his kernel seems to be a little slower he seems to have included the latest drivers which other items relate to data integrity (im reading into the release notes).
NEO: The first thing I did when I got my device install CW, Vegan... Updated Kernels also... Never had an issue until the first time (yes about a day ago) I used the reboot feature of Vegan. That corrupted my user data. I suspect if you have not been performing clean shutdown then you are just lucky. Linux, like any other OS, even with Journaling if you do not perform a clean shutdown you will surely encounter SOME corruption. Typically the corruption is re-mediated by the the file systems integrity controls. You dont even know it happened... 1 in 1000 the integrity controls can not overcome the significant loss of data and thus results in crashes, etc. Some times the corruption happens in areas where are lightly used thus why you would get a Market Reset... that data is easily replaceable on the fly. Core components that require subsystem to run are not replaceable and thus why I had to reformtat. What upsets me is that this failsafe is not working properly most likely as its far too frequent.... I too suspect it has something to do with CW.
But again.. between the wrongly placed power switch, the unprovoked reboots (ie viewsonic screen showing when trying to wake up the device) and the reboot button possibly not performing a proper shutdown will sure increase the chances in a wider distribution of users. So it may not be a CW issue and just some poor design.
When I have time today I will verify if the reboot function performs a clean shutdown... if anyone has the time please post the logcat... Im going to be running around today and will try to get to it..
watson540 said:
shouldnt need source code to debug a dirty shutdown..Cant you just run an adb logcat? maybe run the shutdown command in a terminal on the device and pipe the output into a text file for later viewing
Click to expand...
Click to collapse

stanglx said:
I am very surprised though as the EXT3 filesystem is very resilient to dirty shutdowns (more than EXT4)...
Click to expand...
Click to collapse
AFAIK they're running yaffs ATM. Next move is to ext4...
Read some articles about this several weeks ago, apparently many apps do not properly flush file caches. One of the articles was a Google developer post about file corruption along with their API method which did a cache flush prior to a close, then a bit later was the Google indication that they were planning to move to ext4 FS to further help alleviate the problem.

stanglx said:
I am very surprised though as the EXT3 filesystem is very resilient to dirty shutdowns (more than EXT4)...
I suspect if you have not been performing clean shutdown then you are just lucky. Linux, like any other OS, even with Journaling if you do not perform a clean shutdown you will surely encounter SOME corruption. Typically the corruption is re-mediated by the the file systems integrity controls. You dont even know it happened... 1 in 1000 the integrity controls can not overcome the significant loss of data and thus results in crashes, etc. Some times the corruption happens in areas where are lightly used thus why you would get a Market Reset... that data is easily replaceable on the fly. Core components that require subsystem to run are not replaceable and thus why I had to reformtat. What upsets me is that this failsafe is not working properly most likely as its far too frequent.... I too suspect it has something to do with CW.
Click to expand...
Click to collapse
That's my point. How many times since we've had our Android and smart phones have we had situations where they are turned off or rebooted without the proper procedures? Power drains till they die, they drop and reboot, we clog them up with stuff or some app drives them nuts and they reboot or shut off....Yet you rarely if ever hear about a phone's data being 'corrupted' with stock software. Sure it may happen with official OTAs etc, but never just off-the-bat like what's happening with the G-Tab. But it's not happening to everyone either so I'm just looking to see if there's a pattern.
Even since the G1 and newer phones, you don't really hear about or see file corruption issues on stock software with these phones. It's when users start going to ROMs that you hear of issues cropping up. That's not to say it doesn't happen at all at stock, I just think we're seeing it in a more concentrated fashion here because of all the formatting, re-partitioning, etc. At first you hear, 4GB is a great partition size, then you hear there are problems so move to 2048, then you hear 256MB swap, then no swap since Android doesn't use it. Then dataloop for speed, then no dataloop because of critical issues. Rules and instructions change almost on a daily basis. I think it's more than these poor flash drives can take I find sometimes it's good to keep it simple.
I owned a Vibrant for a while...decided it was a PoS when at stock I was seeing bad lag (because of Sam's terrible FS). People said...do the speedhack, it'll be fast!, but what was the caveat? Having to reboot the phone almost weekly, sometimes several times a week, and people were seeing what? Data corruption. That's not for me. Give me something that is lag free (doesn't have to be a bullet train, just don't skip on video or audio and make sure my live wallpaper and drawer animation is fluid and I'm happy!). Point being....keeping it simple may help to alleviate some of the issues. If people are seeing these problems with stock, then you're absolutely right and it would be a point of contention that the failsafe isn't working right.
Otherwise it seems the stock OS on these things are able to self correct in most situations and it may just be some of the many tweaked features in these ROMs doing something it shouldn't - or, I may just be very lucky indeed.
I'm still dying to get the OTA - I haven't seen one since 3899 yet.

[Q] Data Encryption feature in TF

HI All,
has anyone used the Data Encryption feature on the TF? Are there any known issues? I want to be able to secure my personal data and also my work data stored on my TF.

I was too afraid as every encryption software we have tried in my company has caused performance hits. It's probably the same with the TF.

Kilmar said:
I was too afraid as every encryption software we have tried in my company has caused performance hits. It's probably the same with the TF.
Click to expand...
Click to collapse
That's exactly what I was afraid of too... But there has to be another way to protect my private data and work data...
Mike

I don't think encryption should cause any performance issues. The storage should be decrypted upon boot and then behave normally. That's how laptop full disk encryption works anyway, no performance hit.
The only downside I can see is issues with recovery software etc...
I'm not sure what it encrypts exactly. Maybe just user data? Just the SD card? Full flash?
Sent from my Nexus S using XDA Premium App

csmall said:
I don't think encryption should cause any performance issues. The storage should be decrypted upon boot and then behave normally. That's how laptop full disk encryption works anyway, no performance hit.
The only downside I can see is issues with recovery software etc...
I'm not sure what it encrypts exactly. Maybe just user data? Just the SD card? Full flash?
Sent from my Nexus S using XDA Premium App
Click to expand...
Click to collapse
mind if i ask you where you got this information? it doesn't really make any sense. decrypting an entire drive would be incredibly slow.

finalhit said:
mind if i ask you where you got this information? it doesn't really make any sense. decrypting an entire drive would be incredibly slow.
Click to expand...
Click to collapse
You wouldn't decrypt the entire volume at once though, just what you were trying to access.

You don't actually have to decrypt the disk to access it. It has the key so it can just gain access. The drive is still encrypted, you just can access it. I may not be wording it properly.
Actually decrypting a disk will take time yes. But that isn't the case with accessing it.

that's how i understand encryption works. the password you use merely unlocks the key which is used to decrypt files on demand. of course, the decryption does take a toll on performance, especially on system files that get accessed often.
so it wouldn't be completely without a performance hit.
i do use full encryption on my ubuntu install (i believe including swap) and although it's not noticeable most of the time, i think when things get busy, i does add some overhead.

Android mass storage - g_file_storage.ko

Hello folks. First of all, I have to say that I do not own an Android phone, but
I might be getting a cheap one in the future for some tinkering. And I am simply curious.
So, I need to know some details about the mass-storage mode handling
and it would be cool if you guys could help me out.
Assuming I have rooted my device and created a file with dd or something
and I also set up a loop-device to correspond to this file (basically
a losetup /dev/loop0 /tmp/bla).
Can I expose it into mass-storage mode, like the normal partition where your files lay in? I mean I know from other
smartphone operating systems that this is possible with some shellscripts
which they prepared for this stuff. Those shellscripts basically use the g_file_storage kernel module
for that.
Now I searched the web and found some hints that this could also be the case for android. However,
a friend of mine checked his modules and nope, it seams that it does not exist on his device. I think
it should not not differ from device to device, am I right?.
How is android taking your file partition and enabling it for the usb mass storage mode?
Are there scripts for that? Is it somehow done internally within binaries which use syscalls and stuff like that?
Or: What exactly happens when I plug in the usb cable to the phone and then choose the "enable" button?

hey lawl0r,
i have come to the very same question since i want to try this on the galaxy nexus.
did you get any further with this?

[Q] Encryption and custom recovery

Im just wondering for security reasons im thinking of using androids built in Encryption tool my question is will this effect things such as my custom recovery or other functions and will it have any negative effects on things like battery life or performance

Personally I stay away from encryption. I can't speak for this device but on the s3, if you encrypted sdcard, if you factory reset, that encryption key is lost and that data forever inaccessible. Whether nexus is any better, I don't know.
Recovery will boot fine but since it doesn't have the encryption key, I would assume it cannot browse data that's encrypted. Id love to hear otherwise on this
There will always be an overhead when writing / reading data within the encryption. Extra cycles will be used to actually process the encryption so I expect a slight impact in battery life and a slightly bigger impact to performance
-----------------------
Sent via tapatalk.
I do NOT reply to support queries over PM. Please keep support queries to the Q&A section, so that others may benefit

Is encryption worth it on our device?

I've recently switched to EXT4 on all partitions to be able to run full disk encryption on jgcaap's latest cm13. I've come to find out that our device does indeed suffer a significant performance hit with disk encryption, since the snapdragon 801 doesn't have hardware decryption support like the 810 and 820 do.
On top of that, I've read that Snapdragon's current implementations of encryption have tons of vulnerabilities that allow the attacker to just brute force the password you use to get in. Essentially making encryption much less useful than before.
There's overwhelming evidence that F2FS, even with kernel 3.4, is much faster and better for the storage long term. Do you think its worth it to just switch to F2FS without disk encryption, since there are many ways the authorities could brute force in to my phone anyways?
Your opinions would be greatly appreciated.
-ThunderThighs
Update: for now I've decided to just stick with ext4 and encryption, I've done more research and learned that all of the software encryption vulnerabilities have been parched by google, and my device is ahead of those security patches. As much as I love f2fs, I can't go without storage encryption these days. I've also just discovered the sim card lock feature as well, which ads a layer of encryption on your sim card attached to a pin to protect against theft.