Related
Hi, thought I'd throw this one out here to gather some more general interest, rather than posting it in the streak forum (where there's been some head scratching about it).
I'm trying to work out how we can sensibly extract the update.pkg files that Dell use for the Streak.
I can't see what format the file is, I've opened them with a hexeditor and can't see any obvious file headers.
I've tried various file and pkg utilities but no good.
Any ideas to point me in the right direction to keep looking?
there's a sample rom here
http://mobileupdate.dell.com/LatestBuild/LatestBuild_00.html
The problem
Since HTC introduced Sense 3.5, themers faced a huge problem. The previously used software "M10Tools" wouldn't work with the new version of Sense.
Flemmard and me tried countless hours decoding the new image format, but without any success. The new image format is totally diferent to anything else previously seen.
I made this thread to search for help from all the awesome devs on XDA, hoping that we might find one who can help.
The history
Let me start this with some introduction to the m10 format itself.
The images I am talking about are parts of one big file - the m10 file. We usually have multiple images per m10 file, but the number doesn't really matter.
Together with the raw image data we get a set of meta information. We are not exactly sure what the values mean, but we can guess the meaning from the history of the old, decodable images.
We used to have information like width, height, payload of the image and an integer indicating what kind of image type we have. We know the actual image type for a few of these intergers, but with Sense 3.5, 3.6 and 4.0 HTC added at least two new types.
The facts
We don't have any hard facts for these image types but looking at the "old" image types, we can guess a few things:
The images are in a format the GPU can render directly (Like s3tc, ATC, QTC, etc) (At least this used to be the case, might have changed)
Images are most likely compressed. The ratio between assumed size (based on meta data) and the actual data size indicates some heavy compression. The data itself obviously looks compressed too.
There are no headers or any other help. It is just raw data.
We don't know exactly how the decoded images actually look like, so we can't say what the images display. However, due to latest archievements we "might" know this for images from Sense 3.5 and 3.6 if needed.
The handling software side is all in a few libs and NOT in smali / java, so we can't look for stuff there, however we have the libs, so if someone is pro with assembler he might find out something
I will provide a download which contains several chunks of image data and the according meta data.
If you consider working on this, please do not refrain from thinking about super simple solutions, we worked so long on this that we might be totally confused.
One thing though, this might sound arrogant, but this here really is only for people who have some decent knowledge about file formats, image compression or OpenGL.
The image types
Here is a list of image type we already know ( remember, we don't know where the numbers come from, might be some enum in native code or so)
Type 4: Raw RGB
Type 6: Raw RGBA (still used rather often)
Type 8: ATC RGB (doesn't seem to be used at all anymore)
Type 9: ATC RGBA Explicit (doesn't seem to be used at all anymore)
As you can see we got types WITH and WITHOUT alpha encoding.
Here is the list of UNKNOWN formats:
Type 13 (used way less than type 14, so maybe no alpha?)
Type 14 (this is the most used type, so I assume this one supports alpha encoding)
When thinking about what the data might be, don't throw away crazy ideas like "The data is S3TC /ATC /whatever but compressed again by some 'normal' compression algorithm". Maybe they just replaced type 8 and 9 with an additional compression on top of these types.
The meta data
Okay, so now lets talk about the meta data we get together with the actual data:
We get 4 more or less known chunks of information per image (plus a few unknown things)
Image type (described earlier) (Example: 6)
Image width (Example: 98)
Image height (Example: 78)
A more complex value containing multiple values at once.
Example: "98:78:0:30576"
We used to know the meaning of three of these values. However we are not sure for the new images. Lets explain the old meaning first:
98: Width, same value as the value above
78: Height, same value as the value above
0: It's always 0, we have no idea what it means, but since it's static we didn't care
30576: this used to be the data size. This image has a resolution of 98*78 = 7644 pixels. With a data size of 30576 that means we got 4bytes per pixel.
Lets take a look at the new images now. We still get the same information, however the meaning seems to have changed a bit:
Image type (described earlier) (Example: 14)
Image width (Example: 997)
Image height (Example: 235)
A more complex value containing multiple values at once.
Example: "1000:236:0:118000"
This is the assumed new meaning
1000: Width, but rounded up to a multiple of 4
236: Height, but rounded up to a multiple of 4
0: It's always 0, we have no idea what it means, but since it's static we didn't care
118000: this value is now exactly half of the rounded resolution (1000 * 236 / 2 = 118000)
This would mean only half a byte per pixel. One big problem here: the actual data size does not match this value at all!
The data is way smaller than this value, which indicates that it got compressed a second time
Now lets talk about some very important piece of information: HTC uses the SAME image formats on BOTH a Tegra 3 and a Qualcomm Snapdragon S4.
This obviously means that both Tegra and Snapdragon need to be able to handle this. However, also keep in mind that HTC bought S3 graphics and thefore might got some advantages here.
You can find a statistic on the used formats in the download, it's an Excel sheet with two diagrams showing the usage.
Now this was a long post, I hope someone is still reading this and might have some ideas about what's going on here.
Feel free to ask any questions concerning this.
I am also available in #virtuousrom on Freenode, per PM here or via email: diamondback [at] virtuousrom [dot] com
Download:
The download contains a bunch of unknown images of types 13 and 14 together with their meta data (like explained above)
Download image pack
Solution
After some digging I estimated that the data is compressed with fastlz [0]. Also so if you decompress it, you get exactly Width*Height bytes of data. I dont know the format this data is in, but i guess its the same the uncompressed data (type 8 or 9 or so?) was. Maybe someone could check up on that.
[0] http://fastlz.org/
onlyolli said:
After some digging I estimated that the data is compressed with fastlz [0]. Also so if you decompress it, you get exactly Width*Height bytes of data. I dont know the format this data is in, but i guess its the same the uncompressed data (type 8 or 9 or so?) was. Maybe someone could check up on that.
[0] http://fastlz.org/
Click to expand...
Click to collapse
You are indeed right. We actually found the same a few hours ago What a weird conincidence... :victory:
Type 4 and 6 are changed, they are zipped now too. Which actually breaks backwards compatibility with older Sense versions...
Inside of the zipped data are ETC images, which also explains how they can use the same on S4 and Tegra 3.
The type 14 actually contains TWO images, both ETC. Since ETC doesn't support alpha one is the image and one is an alpha mask...
Funny trick HTC!
Hi !
After the MODEMS thread,
i decided to start this thread to help SGS ACE IIx and S Duos owners in recoveries switching....
So, if you want to flash kk+ roms(Like CM11, BeanStalk...), TWRP is better for you
You can get it >>>>> HERE
If you want to flash stock-based roms(Like PMP Ultra, Kyle/SS/Open, Cosmic...), CWM is better for you
You can get it >>>>> HERE
Just flash your recovery from your current one and then reboot recovery....
Regards......
.rootCoder. said:
Hi !
i decided to start this thread to help SGS ACE IIx and S Duos owners in recoveries switching....
Regards......
Click to expand...
Click to collapse
Thank you for your job!
I'm using TWRP and have already customized .fstab to allow backing up all the partitions
including modem FW and settings (ecrypted, contains real qualcomm efs customized by Samsung incl IMEI, locks and baseband settings) which are very important to be on the safe side and restore device to the working condition independently of what had been damaged and what Samsung and other unfair "money-makers" want me to do or not to do or buy or not to buy. Furthermore, we can dump whole emmc to the raw image incl partition table and secret holes (its almost standard MBR, because emmc is nothing else than almost standard extremely cheap $2 (in retail!) uSD card (the only difference it's soldered as BGA) which sold as some valuable BIG'n'cool "internal storage" and I will not be surprised if its made of MLC NAND which will fail soon to make Samsung happy to sell you a new cool device ). This full emmc dump will be flashable via JTAG as RAW image (RIFF JTAG standard), so we needn't to have $150 RIFF to backup, but can buy it once we would be forced to restore our devices.
S7562 has JTAG connector so even soldering is optional and pin adapter could be used instead.
But there is problem to me - I can't find TWRP recovery settings in external file, so I can't edit primary check boxes positions and can't manage cache entry because it always shows "Android Secure" instead of cache independently of what cache entry contains in .fstab. I realize these settings built-in to the recovery binary so I ask you to share sources in this topic so everyone could be able to fully customize package.
On other hand I would be pleased to cooperate with you because I have 20+years of hacking experience but almost have no android/linux development experience, so I think it could get more time for me to install and tune up environment than hacking itself. I can provide you with full partition layout and some other internals of the Kyle and ready (but not debugged) .fstab to make a few good TWRP assemblies with different fstabs (most full version I'm sure shouldn't be offered to the novices, because they can easily kill their devices) or one more customized and featured.
Furthermore, MAYBE ones (who are familiar with iron and fan) can blow off emmc and solder in standard uSD slot (8-9 wires, I don't think they use SPI pins) to use cheap 16-32Gb uSD or may be superfast 64-128 or even 256Gb for internal storage. Anyway, I can't remember real parts# inside, even if there is MCP instead of pure emmc, ones can always order another BIG'n'Fast emmc (or MCP with emmc and bigger DRAM size), flash it and reball in place. (or reball then flash w/RIFF).
No flashing/software problems they should experience - the only modification req'd is to expand userdata ext4 partition to the size of new emmc or create new additional one and write script to mount it to the system at startup.
Modding is always fun because it's your creature and your win, not the other's labor and mind, bought at the store
The most limiting here is a free time, but I have a small amount.
TheDrive said:
Thank you for your job!
I'm using TWRP and have already customized .fstab to allow backing up all the partitions
including modem FW and settings (ecrypted, contains real qualcomm efs customized by Samsung incl IMEI, locks and baseband settings) which are very important to be on the safe side and restore device to the working condition independently of what had been damaged and what Samsung and other unfair "money-makers" want me to do or not to do or buy or not to buy. Furthermore, we can dump whole emmc to the raw image incl partition table and secret holes (its almost standard MBR, because emmc is nothing else than almost standard extremely cheap $2 (in retail!) uSD card (the only difference it's soldered as BGA) which sold as some valuable BIG'n'cool "internal storage" and I will not be surprised if its made of MLC NAND which will fail soon to make Samsung happy to sell you a new cool device ). This full emmc dump will be flashable via JTAG as RAW image (RIFF JTAG standard), so we needn't to have $150 RIFF to backup, but can buy it once we would be forced to restore our devices.
S7562 has JTAG connector so even soldering is optional and pin adapter could be used instead.
But there is problem to me - I can't find TWRP recovery settings in external file, so I can't edit primary check boxes positions and can't manage cache entry because it always shows "Android Secure" instead of cache independently of what cache entry contains in .fstab. I realize these settings built-in to the recovery binary so I ask you to share sources in this topic so everyone could be able to fully customize package.
On other hand I would be pleased to cooperate with you because I have 20+years of hacking experience but almost have no android/linux development experience, so I think it could get more time for me to install and tune up environment than hacking itself. I can provide you with full partition layout and some other internals of the Kyle and ready (but not debugged) .fstab to make a few good TWRP assemblies with different fstabs (most full version I'm sure shouldn't be offered to the novices, because they can easily kill their devices) or one more customized and featured.
Furthermore, MAYBE ones (who are familiar with iron and fan) can blow off emmc and solder in standard uSD slot (8-9 wires, I don't think they use SPI pins) to use cheap 16-32Gb uSD or may be superfast 64-128 or even 256Gb for internal storage. Anyway, I can't remember real parts# inside, even if there is MCP instead of pure emmc, ones can always order another BIG'n'Fast emmc (or MCP with emmc and bigger DRAM size), flash it and reball in place. (or reball then flash w/RIFF).
No flashing/software problems they should experience - the only modification req'd is to expand userdata ext4 partition to the size of new emmc or create new additional one and write script to mount it to the system at startup.
Modding is always fun because it's your creature and your win, not the other's labor and mind, bought at the store
The most limiting here is a free time, but I have a small amount.
Click to expand...
Click to collapse
Sorry for my longer Delay( coz i'm busy, you know, studying...)
Happy new Christmas year!, and happy new Hijri year !
Thank you for every thing you said
even if you're only a junior 3+ posts member here, you're a one of the most respectable hackers, we may go on and make somthing special with this device.
About the source code of twrp, it's originally owned by codename13, this is only a binary dump and repackaging, he just got the code of kyle(ss)open kernel and made a working recovery, you may contact him for it( coz i didn't find it on his github)
i wish to you a good experience with linux, just like the other systems
and i wish to get some experience from you (coz i still a 16years beginer with some 2years of linux experience )
may god help us.....
Regards......
.rootCoder.
Hello, I have had a look around and so far no real clear answer but a about 6 months ago I had installed in my car a Yessun brand (chinese) 7" headunit. All works reasonably well however it lags so much I will select a song and it will take sometimes about 5 or 6 seconds to select and play. Additionally when playing music from my phone (spotify or the device itself) the metadata doesn't show on the head unit. My question is, is there a way the software (and/or firmware if it meant I can have the bluetooth metadata shared on the head unit) can be updated? Not necessarily by me as I am useless at that stuff but I'd be happy to approach a professional if there was a way this could be done. I just wouldn't even know where to begin to ask or look for someone so the online community is my only choice - thanks in advance
Some info from the website about the head unit:
Operating system - WINCE6.0
Frequency - Cortex-A7
Main chip model - MSB2531
Memory - DDR3-128MHZ
Storage Space - NAND FLASH SLC 256MHZ
SD interface support protocol - SD Memory Card Spec(ver2.0)/MMC Memory card Spec(4.3) Compatible
Navigation software storage - SD card (it has iGO instsalled)
Type of software compilation - ARMV4I / Interface parameter
Communication interface - I2C/SPI/UART/SD
Audio interface - Analog stereo input \ output
Other interface - Common IO/AD/PWM
Do you mean that you wanna to update the WinCE 6.0 or something else?
As far as I know, the latest version for WinCE is Ver. 6.1, which I don't think will lead to a performance improvement for your device.
So, maybe we should seek some approach else.
Hello!
tl;dr
I have an MC70 handheld computer with a DCR7000 magnetic stripe reader attachment running Windows Mobile 5.0. Do you know whether the magnetic stripe reader is able to read cards in RAW format, i.e. getting the bits on the magnetic stripe without any kind of encoding (without character encoding, parity bits, start and end sentinels or LRC)?
I've been trying using Symbol / Motorola EMDK dll libraries for .NET and Zebra DataWedge as well. No luck. There is a RAW_DECODING mode on Symbol dlls but it returns E? when reading any card in an arbitrary encoding.
-------------------------------
For those who love the detail:
Using Motorola EMDK with Symbol API (and the examples provided) I was able to develop a simple .NET application to read vanilla-formated credit cards with no problem. I'm using a custom encoding so no standard card reader would recognise my cards.
I have tried all possible MagStripe libraries in the Motorola EMDK (v2.9) [Symbol.MagStripe.dll, Symbol.MagStripe2.dll, Symbol.MagStripe.Design.dll, Symbol.MagStripe2.Design.dll and their .NET 3.5 upgraded counterparts] as well as the newer DataWedge API (v3.4 for compatibility with my device). While Symbol.MagStripe and Symbol.MagStripe.Design provide means to set the DecoderMode to RAW, the reader always returns "E?" for each track. Maybe I need to change the Track2Format property in some manner --don't really know how to use it--?
The communication between MC70 and DCR7X00 is via serial at 9600 baud, with the following command and response sequence. (in blue, commands sent by MC70; in red, sent by DCR7X00. Newlines added for clarity and sensitive information has been masked with asterisks.)
\x1B !F11001100110001\r
ack\r\n
DCR7000 rev. 1.82\r\n
Format Flags: 11001100110001\r\n
\x1B !S\r
a%B3852************^PLACEHOLDER VORNAME. ^*******************************?
b;3852************=********************?
c%******************==************************************===********************************************?\r
As the above listing shows, the format flags are default to STANDARD_DECODE with NO_LRC to be sent.
When using RAW_DECODE with NO_LRC, the format flags change as follows, but I wasn't able to read any card format. (We use only Track2. We omit Tracks 1 and 3.)
\x1B !F00010001000101\r
ack\r\n
DCR7000 rev. 1.82\r\n
Format Flags: 00010001000101\r\n
\x1B !S\r
bE?\r
The last line on the listing shows how the DCR7X00 recognises that there is data only in track 2, but is not able to decode it (E?) albeit having set the format to RAW_DECODE.
I'm confident that RAW decoding has to function properly, as I know of commercial stores using RAW decoding with the same hardware as mine. Moreover, in the DecoderModes Enumeration entry inside the EMDK Help file (EMDKfordotNET.chm):
Raw Data Decoder. The Raw Data Decoder sends magnetic data in raw data format so the application program can perform complicated decoding. With this feature, raw data, which are bit level data in the card, can be sent to the application program for further processing. Two ASCII characters represent each raw data byte. Track selection is invalid for the raw data decoder; that is, all encoded data of three tracks in the card is sent.
Click to expand...
Click to collapse
Finally, DataWedge has also been installed in the MC70 but after creating a profile with the DCR using library MSRMC50.dll it has trouble recognising the DCR7X00. When starting the DataWedge service in its application, it remains to IDLE. I discarded the DataWedge option after seeing the following line in the (current, not v3.4) DataWedge documentation at Zebra TechDocs:
For modifying the acquired data, DataWedge offers only the formatting options shown in the sections below. Options for formatting data beyond those shown can be found in the ID TECH SecureHead User Manual. Custom data formatting is the sole responsibility of the developer.
Click to expand...
Click to collapse
But ID TECH SecureHead User Manual describes a magnetic head different from the one used inside the DCR7X00 as this one encrypts data directly on the head providing Serial or USB interfaces while the one inside DCR7X00 contains seven used connections to the three different coils of the read/write head. Then the secure microcontroller inside the DCR7X00 responds with a serial command to the MC70.
-------------------------------
Thank you very much for your time and answers .
John