Related
Ok, I've seen numerous questions about the app called HTC Performance & have disassembled the executable. While my knowledge of these thing is by no means great, I have found some very interesting functions.
Maybe someone with more reverse engineering & code experience can take a look, but with IDA Pro there are some very interesting functions & strings.
Some of the calls & code are deprecated & no longer used in WM6 + but some of them are.
It is possible, especially for evB equiped roms, that this prgram acts like a server of sorts for some programs & processes. But being as it is initiated with Smartphone only functions I doubt it.
some of the more interesting functions in the HTC Performance app are:
SHInitExtraControls Which appears to be for Smartphone only
GetSystemMetrics WM6 Pro valid - Gets System Width & Heigth in pixels. Posible uses include program optimization based on the appropriate pixel returns
CreateMutexW - coredll - used to connect to core via net cf for obtaining device info- Usually eVB related apps use to call coredll info
memmove = takes more memory than memcpy but may be used to ensure unicode strings not used on odd memory addresses, this could increase speed on apps that incorrectly do this.
InterlockedCompareExchange, InterlockedDecrement, InterlockedExchange, InterlockedExchangeAdd, and InterlockedIncrement = functions provide a simple mechanism for synchronizing access to a variable that is shared by multiple threads. The threads of different processes can use this mechanism if the variable is in shared memory.
InterlockedCompareExchange = function performs an atomic comparison of the Destination value with the Comperand value. If the Destination value is equal to the Comperand value, the Exchange value is stored in the address specified by Destination. Otherwise, no operation is performed
YAXPAX = can speed up access of written C++ Code
ReleaseMutex = Mutex functions are used to release shared functions
EnumWindows = (..) to execute a task. EnumWindows (..) enumerates through all existing windows (visible or not) and provides a handle to all of the currently open top-level windows. Each time a window is located, the function calls the delegate (named IECallBack) which in turn calls the EnumWindowCallBack (..) function and passes the window handle to it. Not sure howthis is used though.
LoadAcceleratorsW = ??? Appears to be old CE function. Deprecated???
realloc = String Optimization
malloc = RAM Allocation
GetDeviceCaps = gets dev info, can be used to the optimize redraw based on device constraints already known
LocalReAlloc = This function changes the size or the attributes of a specified local memory object. The size can increase or decrease.
EnterCriticalSection = The threads of a single process can use a critical section object for mutual-exclusion synchronization. The process is responsible for allocating the memory used by a critical section object, which it can do by declaring a variable of type CRITICAL_SECTION. can grant exclusive access to memory
ReleaseDC = This function releases a device context (DC), freeing it for use by other applications. The effect of ReleaseDC depends on the type of device context.
Again, I am not a programmer, I know a few things, & am pretty competent with the lower operations of firmware, but the rest of the CE code is not my cup of tea. There are many more functions in HTC Performance. These are only a few functions found after a brief 20 minute peak.
But maybe, maybe, some of the function calls can help us to understand if this app can be moddified to properly function on the Kaiser.
It is possible that on some evB enabled apps, that maybe some of the HTC Performance app are retained & possibly function, that is pure speculation though, & again I doubt it.
Any CE code experts out there wanna take a look? I have, & based on what I've seen, I'll have to say FICTION!
Info
Hi,
Since I haven't really had time to see whats new and all I haven't the foggiest idea what HTC Performance is/what it is supposed to do.
But I can tell you that the functions you listed are not special in any way. Most of them would appear in every application that displays anything on the screen. For instance getting system metrics is required for any application displaying scroll bars, etc. All the interlocked and critical section stuff is just thread synchronization.
But that's OK, the use of windows APIs really doesn't mean much, other than the application runs on Windows...its the non-API stuff that defines an application. If the application you're looking at writes changes to registry keys, etc. you may want to look into that as those would be the lasting changes to the device.
Cheers,
Why is there concurrency related stuff in there? Surely that should all be handled by the operating system, rather than a running application? (That said, most of my concurrency knowledge is either theoretical or based at a high level, so I could be wrong here).
High Performance Cab
You can also check this thread...
http://forum.xda-developers.com/showthread.php?t=366792
Quentin- said:
Hi,
Since I haven't really had time to see whats new and all I haven't the foggiest idea what HTC Performance is/what it is supposed to do.
But I can tell you that the functions you listed are not special in any way. Most of them would appear in every application that displays anything on the screen. For instance getting system metrics is required for any application displaying scroll bars, etc. All the interlocked and critical section stuff is just thread synchronization.
But that's OK, the use of windows APIs really doesn't mean much, other than the application runs on Windows...its the non-API stuff that defines an application. If the application you're looking at writes changes to registry keys, etc. you may want to look into that as those would be the lasting changes to the device.
Cheers,
Click to expand...
Click to collapse
No, registry would not necesarily be the place to look. For this application the registry will only report whether or not the App is running or not. It is supposed to be a speed optimization application. My thought were that it could possibly be acting as a server of sorts, handling some thread optimization & resource allocation. Correct though, most of those API's are importing device info, beyond that, I am lost as to how it handles it, if it does at all. That said, there are many things that don't show up in the registry & many things can't be altered via the registry b'c they are set or handled before initialization or loading of the registry, possibly thru the OAL. Even tougher to say in a two chip device with as little known info as the msm7xxx processors. If anyone with real coding knowledge could take a look at the executable & see just what it's doing with the info, that would be great.
dperren said:
Why is there concurrency related stuff in there? Surely that should all be handled by the operating system, rather than a running application? (That said, most of my concurrency knowledge is either theoretical or based at a high level, so I could be wrong here).
Click to expand...
Click to collapse
That is indeed the center of my question & also what leads me to question how the app functions. Is it playing a role in thread priority optimization, & possibly redraw based on the polls, or is it just a partially gutted application miising a ton of registry data that never worked?
Just wondered what is the long term future for the Java-ish language on android given the current case. I wrote a bit of part satire, part idea at the following address.
http://my.opera.com/jackokring/blog/java-oracle-android-and-google
Does anyone else recognize the immense opportunity opening for a new language across the mobile market for a new syntax, and rationalized smaller library?
More on supa
Apart from the following alterations to the base language and the ones proposed earlier, I'd like some feedback on what should go into the libraries...
I'd also remove short, char and byte as types, and have a PackedArray class for such needs. To muddy the base type system with anything less than an int maybe is an obscure homage to history. I'd then be left with just int and long and object pointer and void.
Any other types would be library based. For example String.
The scoping rules being strict relates to SMP. For recursive functions, the extra need to pass some extra parameters, to get extra locals is not a bad idea, as it forces an understanding that automatic synchronization, would already enforce. And would lead to simpler recursive functions using the instance variables as type 'static in the c function sense' for data pass back to the single writer previous recursion instance.
Now if your thinking how would the usual ijk for loop variables be defined within multiple methods, and not generate a 'multiple writer method error' this is just syntax, and forcing actually descriptive variable names is surely a good thing. The fact that the data segment would grow in size, as the method instance variable set grew would be more of an issue. But then again, I never suggested removing the word 'transient'. After all if a stack frame local is not transient then what is? The word volatile also becomes redundant, as all class variables are volatile.
The keyword strictfp is also useless in supa.
The Kernel Source Saga
aka
How Samsung Is Continuing to Violate the GPL
As you are all aware, there are still no custom kernels or any functioning custom recoveries for this fine (despite its flaws) device - the Samsung Galaxy Indulge (specifically, the SCH-R910). We'd love to be able to say that's not the case, but it is - and will continue to be, for as long as the issues outlined below continue. Samsung has left us not just in the lurch, but completely screwed - whether it is simple incompetence, disregard for the prepaid device community, interference by MetroPCS, or flat-out malice, the outcome is the same.
The Symptoms
The problem here is not that Samsung has not published some sort of source package. They have done at least that - enough to keep most observers happy. However, this package was broken from the start. The first package that Samsung published for the R910 included a badly-broken netfilter, which stumped our kernel experts for a good while before the decision was made to simply substitute the three pieces of netfilter from another device.
Doing this allowed our kernel developers (and myself) to compile a kernel, linked with a clean initramfs extracted from the stock kernel. In a properly-released source package, this should be enough - but it wasn't. These kernel attempts would appear to boot, but about 10-15s after completion of the boot animation, would die completely. The LTE and CDMA baseband versions, in About Phone, would display as Unknown, if one could get there quickly enough.
Three of the top Epic kernel developers, myself, and a long-time Android veteran analyzed kmsg logs (see below) and other factors, and determined that it was impossible to generate what is in the stock binary blob from the source package published by Samsung.
Warning: technical details! The following IRC log summarizes the issue:
[10:29] <@k0nane> The kernel now builds successfully - no more errors in netfilter, out of the box - but with the stock initramfs, it will exhibit the same behavior as previous builds. After boot, it will shut down within 10-15s. If there is time to get to About Phone, it will show the baseband versions as CDMA: unknown and LTE: unknown.
[10:30] <@k0nane> The modules that are built, which is not the complete set as in the stock initramfs, are /significantly/ larger than the stock copies and cannot be put in a kernel build as even with LZMA compression it will be far too large to fit in bml8.
[10:31] <@k0nane> So as before, if the solution is to use the (again, incomplete) newly-compiled modules, it's no solution at all. Thus there is still no way to build a kernel identical to the stock build.
[see logs below]
[10:36] <@k0nane> Expert analysis caught one of the anomalies:
[10:36] <@k0nane> <@Decad3nce> DRockstar: <4>[ 6.042947] [MULTIPDP][poll_thread] dpram_open failed!!
[10:36] <@k0nane> <@Decad3nce> probe never goes through
[10:36] <@k0nane> <@Decad3nce> and you fails
[10:36] <@k0nane> <@Decad3nce> :x
[10:36] <@k0nane> <@Decad3nce> what should happen (from start)
[10:36] <@k0nane> <@Decad3nce> <4>[ 6.275749] [MULTIPDP] dpram_open successd !!!!
[10:36] <@k0nane> <@Decad3nce> <3>[ 6.275770] [IDPRAM] <dpram_tty_ioctl:1718> IOCTL cmd = 0x5405
[10:36] <@k0nane> <@Decad3nce> <3>[ 6.280198] [IDPRAM] <dpram_tty_ioctl:1718> IOCTL cmd = 0x540
[10:36] <@k0nane> <@Decad3nce> which triggers phone active
[10:36] <@k0nane> <@Decad3nce> <3>[ 6.757528] [IDPRAM] <phone_active_irq_handler:2195> PHONE_ACTIVE level: LOW , phone_sync: 0
[10:36] <@k0nane> <@Decad3nce> then you have your power on, etc
[10:36] <@k0nane> <@Decad3nce> 3>[ 9.870551] [IDPRAM] dpram_phone_power_on() ...
[10:36] <@k0nane> <@Decad3nce> <3>[ 10.450069] [IDPRAM] <phone_active_irq_handler:2195> PHONE_ACTIVE level: LOW , phone_sync: 0
[10:36] <@k0nane> <@Decad3nce> <3>[ 10.457065] [IDPRAM] <phone_active_irq_handler:2195> PHONE_ACTIVE level: HIGH, phone_sync: 0
[10:36] <@k0nane> <@Decad3nce> also, not sure what this is
[10:36] <@k0nane> <@Decad3nce> <4>[ 6.042931] [MULTIPDP]filp_open() failed~!: -6
[10:36] <@k0nane> <@Decad3nce> doesn't occur in your goodlog
[10:36] <@k0nane> <@DRockstar> hmm, what could prevent the probe?
[10:36] <@k0nane> <@Decad3nce> the dpram_open failing
[10:37] <@k0nane> And to summarize that, "so from boot, goes through kernel init, then device init, then when it gets to the radio interface layer.. it dies a horrible death because of no access to dpram0"
[10:37] <@k0nane> The details should be unimportant, however, if what's provided is identical to/can produce stock (which, as we see, it cannot).
[10:39] <@k0nane> http://k0nane.info/size.png <== a comparison of stock (left) vs. compiled (right) module sizes
Click to expand...
Click to collapse
Logs: Bad Boot #1 Bad Boot #2 Clean Boot
Source and initramfs as of these logs: Github
What Has Samsung Done?
I have been in contact with SamsungJohn (John Imah), Samsung's official developer representative on Twitter. On first notification of the issue, no action had been taken, and it took an RT campaign to catch Mr. Imah's eye. Approximately one week later, Mr. Imah joined me in IRC, and took note of the information above. Shortly after that, Samsung released a new source package on opensource.samsung.com. That was great - except that the only difference was an end to the out-of-the-box compile errors. Everything else about this source package was identical. The results were the same.
Within the next business day, Mr. Imah requested that I send the information to his corporate email address, as he had an "HQ team" there with him. Unfortunately, that was June 22. Mr. Imah has not responded to tweets or DMs since then, and we have been given absolutely no indication that any work has been done on this issue. The source package has not been updated again.
I have given Samsung ample notification and time to work with myself and my fellow developers to resolve this issue without a public expose. They have failed to so much as maintain the appearance of progress, let alone make any. It was, is, time for the saga to end. Samsung is hereby on notice. They, by failing to provide the source necessary to generate what is included in the stock kernel binary, are violating the General Public License.
http://www.gnu.org/copyleft/gpl.html said:
You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways:
- a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange.
- b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge.
- c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b.
- d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.
- e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d.
Click to expand...
Click to collapse
How Can You Help?
Tweet, email, and otherwise contact every public-facing Samsung representative.
Tell your friends. Have them do the same.
Send a copy of this report to [email protected] - the source in question is the Linux kernel, which is owned by the Free Software Foundation.
Just spread the word - until Samsung feels the heat from this (feel free to contact their legal department!) we will not see results. Development for the Indulge will continue to be held back. Only the users will suffer.
--
Follow me on Twitter @k0nane/@publik0.
x-posted from AndroidForums.
for anyone that thinks this isnt worthwhile, imagine if your device had no source.. where would your kernels be? imagine if this became the trend, to give out crap source
shabbypenguin said:
for anyone that thinks this isnt worthwhile, imagine if your device had no source.. where would your kernels be? imagine if this became the trend, to give out crap source
Click to expand...
Click to collapse
And on top of that, I have every reason to believe - though IANAL - their behavior is illegal.
RT the thread.
I blew this up over on samsungs facebook page.. someone got pissed and deleted my post..
http://www.facebook.com/SamsungMobileUSA
I started to 'comment' on a bunch of their stuff w/ a link to this thread..
I might get my account suspended BWAHAHAHAHA..
bastards
Let's call out @samsungjohn on twitter!
Sent from my Epic™ 4G using XDA Premium app
This is the vary reason i will NEVER BUY any thing sumsuck ever makes, EVER AGAIN!
I have a suspicion that what's going on here is actually just a (possibly long-standing) bug. However, the situation is a bit strange all around.
To be specific, I suspect it's a race condition that happens to be triggered by your kernel build, but not Samsung's. This could easily be due to nothing more than using a different compiler, i.e., one that produces more efficient code.
I notice that in "Bad Boot #1" and "Bad Boot #2", the "[MULTIPDP][poll_thread] dpram_open failed!!" message is preceeded by "[MULTIPDP]filp_open() failed~!: -6". Looking at multipdp.c, dpram_open is failing to open "/dev/dpram1" and returning ENXIO ("No such device or address"), indicating that /dev/dpram1 doesn't exist yet.
Now, there's only two references to "/dev/dpram1" in the entire stock kernel image. The first is the line in multipdp.c that it's failing on. The second is in initramfs's /sbin/init. Here, Android's init actually contains a simplified implementation of udev to handle kernel uevents. That is, it's responsible for dynamically creating device nodes as kernel drivers are loaded, one of which is /dev/dpram1.
What I think is happening is this:
init.rc insmods dpram.ko
dpram.ko registers the dpram tty devices, issuing an add uevent message for device 252:2.
... (some time elapses) ...
init processes the uevent message, creates /dev/dpram1.
Meanwhile:
init.rc insmods multipdp.ko
multipdp.ko creates the poll_thread kernel thread.
poll_tread calls dpram_open.
dpram_open attempts to open /dev/dpram1, which might not exist yet.
Probably what's happening is that Samsung's kernel "takes long enough" to get to the "insmod multipdp.ko" part of init.rc that init has time to process the dpram.ko uevents and create /dev/dpram1, so that by the time dpram_open is called, the device node exists. However, your kernel is "fast enough" that init.rc gets to "insmod multipdp.ko", creates the new kernel thread and executes it, before init has caught up on device node creation. Thus, /dev/dpram1 doesn't exist and everything goes to ****.
Now here is where it gets bizarre. If we compare against the Epic's EC05 init.rc, there's an interesting stanza prior to module initialization that's missing here:
Code:
#samsung dpram modules
mknod 0666 /dev/dpram0 c 255 1
mknod 0666 /dev/dpram1 c 255 2
mknod 0666 /dev/dpramerr c 255 0
mknod 0666 /dev/ttyCDMA0 c 240 1
Presumably this is supposed to create the missing device nodes before insmoding both dpram.ko and multipdp.ko, eliminating the race condition. However, mknod is a command not supported by init, there's no mknod program in the initramfs, and those are the wrong device numbers. Perhaps it's an acknowledgment that the race condition exists on both devices, even though nothing is actually done on the Epic to avoid it? It's just, ... strange.
Unfortunately I can't think of a great solution. One option would be to modify multipdp.c to add a sleep loop around "filp_open(DPRAM_DEVNAME, ...)" to wait for the device to be created. However, that would mean using your own compile of multipdp.ko instead of the one Samsung provides, which could open a different can of worms.
A second option is to move "insmod /lib/modules/multipdp.ko" until after mounting /system, hoping that init would've caught up by then. This can even be insured by running a script that checks for the device node and sleeps if it doesn't exist yet. Something like:
Code:
#!/system/bin/sh
for i in 1 2 3 4 5; do
ls /dev/dpram1 && break
sleep 1
done
and call it from init.rc, after mounting /system, with:
Code:
exec /system/etc/wait_for_dpram1.sh
Anyways, even if this particular issue turns out not to be the culprit, I wouldn't be too quick to conclude that Samsung must've released bad/wrong/old source code. In general, building code with a different compiler, or even a different compiler version, can shake out unfortunate bugs like this. I really hate to ruin any good-will between Samsung and the community by making inflammatory acusations when, frankly, they're probably just as puzzled as we are over this.
Best of luck!
mkasick said:
I have a suspicion that what's going on here is actually just a (possibly long-standing) bug. However, the situation is a bit strange all around.
To be specific, I suspect it's a race condition that happens to be triggered by your kernel build, but not Samsung's. This could easily be due to nothing more than using a different compiler, i.e., one that produces more efficient code.
I notice that in "Bad Boot #1" and "Bad Boot #2", the "[MULTIPDP][poll_thread] dpram_open failed!!" message is preceeded by "[MULTIPDP]filp_open() failed~!: -6". Looking at multipdp.c, dpram_open is failing to open "/dev/dpram1" and returning ENXIO ("No such device or address"), indicating that /dev/dpram1 doesn't exist yet.
Now, there's only two references to "/dev/dpram1" in the entire stock kernel image. The first is the line in multipdp.c that it's failing on. The second is in initramfs's /sbin/init. Here, Android's init actually contains a simplified implementation of udev to handle kernel uevents. That is, it's responsible for dynamically creating device nodes as kernel drivers are loaded, one of which is /dev/dpram1.
What I think is happening is this:
init.rc insmods dpram.ko
dpram.ko registers the dpram tty devices, issuing an add uevent message for device 252:2.
... (some time elapses) ...
init processes the uevent message, creates /dev/dpram1.
Meanwhile:
init.rc insmods multipdp.ko
multipdp.ko creates the poll_thread kernel thread.
poll_tread calls dpram_open.
dpram_open attempts to open /dev/dpram1, which might not exist yet.
Probably what's happening is that Samsung's kernel "takes long enough" to get to the "insmod multipdp.ko" part of init.rc that init has time to process the dpram.ko uevents and create /dev/dpram1, so that by the time dpram_open is called, the device node exists. However, your kernel is "fast enough" that init.rc gets to "insmod multipdp.ko", creates the new kernel thread and executes it, before init has caught up on device node creation. Thus, /dev/dpram1 doesn't exist and everything goes to ****.
Now here is where it gets bizarre. If we compare against the Epic's EC05 init.rc, there's an interesting stanza prior to module initialization that's missing here:
Code:
#samsung dpram modules
mknod 0666 /dev/dpram0 c 255 1
mknod 0666 /dev/dpram1 c 255 2
mknod 0666 /dev/dpramerr c 255 0
mknod 0666 /dev/ttyCDMA0 c 240 1
Presumably this is supposed to create the missing device nodes before insmoding both dpram.ko and multipdp.ko, eliminating the race condition. However, mknod is a command not supported by init, there's no mknod program in the initramfs, and those are the wrong device numbers. Perhaps it's an acknowledgment that the race condition exists on both devices, even though nothing is actually done on the Epic to avoid it? It's just, ... strange.
Unfortunately I can't think of a great solution. One option would be to modify multipdp.c to add a sleep loop around "filp_open(DPRAM_DEVNAME, ...)" to wait for the device to be created. However, that would mean using your own compile of multipdp.ko instead of the one Samsung provides, which could open a different can of worms.
A second option is to move "insmod /lib/modules/multipdp.ko" until after mounting /system, hoping that init would've caught up by then. This can even be insured by running a script that checks for the device node and sleeps if it doesn't exist yet. Something like:
Code:
#!/system/bin/sh
for i in 1 2 3 4 5; do
ls /dev/dpram1 && break
sleep 1
done
and call it from init.rc, after mounting /system, with:
Code:
exec /system/etc/wait_for_dpram1.sh
Anyways, even if this particular issue turns out not to be the culprit, I wouldn't be too quick to conclude that Samsung must've released bad/wrong/old source code. In general, building code with a different compiler, or even a different compiler version, can shake out unfortunate bugs like this. I really hate to ruin any good-will between Samsung and the community by making inflammatory acusations when, frankly, they're probably just as puzzled as we are over this.
Best of luck!
Click to expand...
Click to collapse
.....and why are you not a recognized developer? You deserve it more than me.
mkasick, thanks for the reply, we will look into this. EDIT: Keep in mind that we are using the same toolchain as Samsung, specifically CodeSourcery 2009q3.
mkasick said:
I have a suspicion that what's going on here is actually just a (possibly long-standing) bug. However, the situation is a bit strange all around.
To be specific, I suspect it's a race condition that happens to be triggered by your kernel build, but not Samsung's. This could easily be due to nothing more than using a different compiler, i.e., one that produces more efficient code.
I notice that in "Bad Boot #1" and "Bad Boot #2", the "[MULTIPDP][poll_thread] dpram_open failed!!" message is preceeded by "[MULTIPDP]filp_open() failed~!: -6". Looking at multipdp.c, dpram_open is failing to open "/dev/dpram1" and returning ENXIO ("No such device or address"), indicating that /dev/dpram1 doesn't exist yet.
Now, there's only two references to "/dev/dpram1" in the entire stock kernel image. The first is the line in multipdp.c that it's failing on. The second is in initramfs's /sbin/init. Here, Android's init actually contains a simplified implementation of udev to handle kernel uevents. That is, it's responsible for dynamically creating device nodes as kernel drivers are loaded, one of which is /dev/dpram1.
What I think is happening is this:
init.rc insmods dpram.ko
dpram.ko registers the dpram tty devices, issuing an add uevent message for device 252:2.
... (some time elapses) ...
init processes the uevent message, creates /dev/dpram1.
Meanwhile:
init.rc insmods multipdp.ko
multipdp.ko creates the poll_thread kernel thread.
poll_tread calls dpram_open.
dpram_open attempts to open /dev/dpram1, which might not exist yet.
Probably what's happening is that Samsung's kernel "takes long enough" to get to the "insmod multipdp.ko" part of init.rc that init has time to process the dpram.ko uevents and create /dev/dpram1, so that by the time dpram_open is called, the device node exists. However, your kernel is "fast enough" that init.rc gets to "insmod multipdp.ko", creates the new kernel thread and executes it, before init has caught up on device node creation. Thus, /dev/dpram1 doesn't exist and everything goes to ****.
Now here is where it gets bizarre. If we compare against the Epic's EC05 init.rc, there's an interesting stanza prior to module initialization that's missing here:
Code:
#samsung dpram modules
mknod 0666 /dev/dpram0 c 255 1
mknod 0666 /dev/dpram1 c 255 2
mknod 0666 /dev/dpramerr c 255 0
mknod 0666 /dev/ttyCDMA0 c 240 1
Presumably this is supposed to create the missing device nodes before insmoding both dpram.ko and multipdp.ko, eliminating the race condition. However, mknod is a command not supported by init, there's no mknod program in the initramfs, and those are the wrong device numbers. Perhaps it's an acknowledgment that the race condition exists on both devices, even though nothing is actually done on the Epic to avoid it? It's just, ... strange.
Unfortunately I can't think of a great solution. One option would be to modify multipdp.c to add a sleep loop around "filp_open(DPRAM_DEVNAME, ...)" to wait for the device to be created. However, that would mean using your own compile of multipdp.ko instead of the one Samsung provides, which could open a different can of worms.
A second option is to move "insmod /lib/modules/multipdp.ko" until after mounting /system, hoping that init would've caught up by then. This can even be insured by running a script that checks for the device node and sleeps if it doesn't exist yet. Something like:
Code:
#!/system/bin/sh
for i in 1 2 3 4 5; do
ls /dev/dpram1 && break
sleep 1
done
and call it from init.rc, after mounting /system, with:
Code:
exec /system/etc/wait_for_dpram1.sh
Anyways, even if this particular issue turns out not to be the culprit, I wouldn't be too quick to conclude that Samsung must've released bad/wrong/old source code. In general, building code with a different compiler, or even a different compiler version, can shake out unfortunate bugs like this. I really hate to ruin any good-will between Samsung and the community by making inflammatory acusations when, frankly, they're probably just as puzzled as we are over this.
Best of luck!
Click to expand...
Click to collapse
You weed out toolchain, source, and what you have left is a defconfig that probably has more debugging active than the one that's shipped with the source.
No other way to explain size differential.
EDIT: Also, holy awesome on that analysis.
mkasick said:
One option would be to modify multipdp.c to add a sleep loop around "filp_open(DPRAM_DEVNAME, ...)" to wait for the device to be created.
Click to expand...
Click to collapse
Attached is a patch that should implement this. I've also attached a build of the kernel with it applied--given that we might be looking at funny race condition issues, it's probably worth seeing if the build environment has as much of an effect here.
Mind you, I don't have an Indulge and was unable to test. No idea if this actually works, but hopefully it's progress.
Now, if it does work, it's worth running a dmesg and looking for a "/dev/dpram1 does not exist yet, sleeping." just prior to "dpram_open successd !!!!". If that message isn't there, then it wasn't strictly the patch that contributed to a successful kernel, rather it's probably an artifact of the build.
Full disclosure:
The kernel sources are from SCH-R910_OpenSource.zip posted the Samsung open source page. The initramfs was extracted from this Odin repackage of the stock kernel, linked to from androidforums, although in retrospect I probably should've pulled it from k0nane's GitHub. It was compiled with CodeSourcery G++ Lite 2010q1-188 (EABI) on a Debian sid machine last updated sometime two weeks ago.
In additon to the aforementioned patch (indulge_multipdp_wait_for_dpram1.diff), I used another patch (indulge_no_ifdef_linux.diff) to get rid of the single "#ifdef linux" that trips up the eabi compiler and shouldn't be necessary with the gnueabi one. Otherwise, the only other source changes I made were to the CROSS_COMPILE variable in Kernel/Makefile and CONFIG_INITRAMFS_SOURCE in Kernel/arch/arm/configs/forte_01_defconfig to set the appropiate paths for those two.
I build the kernel once with the original initramfs, copied the newly-built modules/samsung/multipdp/multipdp.ko over the existing one in the initramfs, and rebuilt the kernel to link in the updated initramfs. That should be it.
k0nane said:
Keep in mind that we are using the same toolchain as Samsung, specifically CodeSourcery 2009q3.
Click to expand...
Click to collapse
Yes you're right.
I know that for the Epic's DI18 Samsung used what looked to be an in-house build of GCC 4.3.1 for the stock kernel even though they were recommending CodeSourcery in the source README. Apparently they switched to 2009q3-67 for Froyo and I never noticed.
Decad3nce said:
You weed out toolchain, source, and what you have left is a defconfig that probably has more debugging active than the one that's shipped with the source.
No other way to explain size differential.
Click to expand...
Click to collapse
The defconfig on my Github is identical to that of the package on opensource.samsung.com with the exception of LZMA compression being enabled and a path to the initramfs files being set.
mkasick said:
Attached is a patch that should implement this. I've also attached a build of the kernel with it applied--given that we might be looking at funny race condition issues, it's probably worth seeing if the build environment has as much of an effect here.
Click to expand...
Click to collapse
I'll pass this on to an Indulge user, or test it myself if I find a working microUSB cable first. Thanks. EDIT: No such luck! Died within 15s like most kernel builds do. Unfortunate. I can have the user acquire a log, but I am 95% certain no new info will be presented.
k0nane said:
Died within 15s like most kernel builds do.
Click to expand...
Click to collapse
Crud.
k0nane said:
I can have the user acquire a log, but I am 95% certain no new info will be presented.
Click to expand...
Click to collapse
I'd be curious if it contains any more statements about dpram_open, suggesting whether or not that patch had any effect. Or the built multipdp.ko even works.
Either way, if the device is automatically rebooting after 15 seconds, it's either hitting a kernel panic or something funny is going on with the watchdog. If we could get console output with a UART jig that would likely be helpful.
Can (stock) recovery be booted with a source-compiled kernel and survive for some length of time?
mkasick said:
I'd be curious if it contains any more statements about dpram_open, suggesting whether or not that patch had any effect. Or the built multipdp.ko even works.
Click to expand...
Click to collapse
I will acquire a log ASAP. A new copy of that kernel with adbd flipped to root on boot would be useful, as it's difficult to capture a full kmsg without it - I can generate it via your patch, or if you wish to substitute in the initramfs from my github, that's an option as well.
mkasick said:
Either way, if the device is automatically rebooting after 15 seconds, it's either hitting a kernel panic or something funny is going on with the watchdog. If we could get console output with a UART jig that would likely be helpful.
Click to expand...
Click to collapse
And unfortunately, building one is a bit beyond my skill level. I'd be willing to buy one. The jig should work for the Indulge, given it is essentially a Galaxy S device. I agree that it's most likely panicking - it would be helpful if there were some kind of output to the display as there is with Epic kernel panics.
mkasick said:
Can (stock) recovery be booted with a source-compiled kernel and survive for some length of time?
Click to expand...
Click to collapse
That has not yet been tested. There's a /system/bin/recovery and a recovery partition; as with the Epic and other devices, the partition should be triggered by the button combination on boot. I'll have a user flash a built kernel to bml9.
k0nane said:
or if you wish to substitute in the initramfs from my github, that's an option as well.
Click to expand...
Click to collapse
Attached is a build with your github initramfs.
k0nane said:
And unfortunately, building one is a bit beyond my skill level. I'd be willing to buy one. The jig should work for the Indulge, given it is essentially a Galaxy S device.
Click to expand...
Click to collapse
I have to read through that thread again, but I believe the simple version of the JIG is a 3.3v USB-to-serial converter along with a single resistor to activate the UART. I have some FT232 converters lying around, but they're 5v.
I'll order some parts and see if I can build one for myself in a week or so, as it would be useful for Epic debugging. If it works, we can work something out to get one sent your way.
k0nane said:
That has not yet been tested. There's a /system/bin/recovery and a recovery partition; as with the Epic and other devices, the partition should be triggered by the button combination on boot.
Click to expand...
Click to collapse
I'm thinking ~15 seconds is a somewhat strange time to die during boot. If recovery (with adb) runs stable, then the kernel works in a minimal environment. Perhaps some narrowing down of the boot sequence can be done from there?
Sorry to derail this but I'm cross posting here..
I sent an inquiry to the EFF to see if this is an issue that they can help with. Below is the response I received:
Rebecca said:
Rebecca S. Reagan [email protected] to me
show details 6:27 PM (12 hours ago)
Hi there!
Thank you for contacting the Electronic Frontier Foundation (EFF).
Unfortunately, we do not have the capacity necessary to take on open source issues like this one. The Electronic Frontier Foundation (EFF) is a small, grassroots legal advocacy non-profit focused on defending civil liberties and constitutional rights online with very limited resources. As such, we have a very important but narrowly focused mission and are not equipped to address all of the potential issues that escape our limited scope.
Regards,
Rebecca
Click to expand...
Click to collapse
I took a second look, found, and fixed another problem. Attached are more builds.
I compiled an unpatched kernel using 2009q3-67, decompressed it, and ran "strings" against both it and the stock kernel. I diffed the strings output to look for anything missing that I could grep for in the source tree. I found some things in "Kernel/arch/arm/mach-s5pv210/mach-aries.c" in the stock kernel that were missing from my build.
First were missing definitions for "dpram-device", "onedram", and "modemctl". The second were strings corresponding to the modemctl_cfg_gpio function, which is responsible for setting the "PHONE_ON", "CP_RST", and "PDA_ACTIVE" gpios. For these items to be included requires that CONFIG_SAMSUNG_PHONE_TTY and CONFIG_SAMSUNG_PHONE_SVNET be set in the kernel config file. Which they are (as "=m") in forte_01_defconfig.
Here's the problem: when build_kernel.sh runs "make defconfig", the both config options are missing in the resulting .config file. It turns out that as part of the reorganization that Samsung did to package the source release, they moved the onedram drivers from the Kernel source directory to the external module one, and dropped the Kconfig files that define these options. Without them, the build infrastructure assumes that all (useful) references those config options are removed from the all source files and purges them from the configuration. Except they're still needed by mach-aries.c, and without them, a key part to the kernel radio interface goes missing.
This definitely contributes to why the radio can't be brought up on boot. This may also cause the reboot after 15 seconds, although I'm not certain.
Anyways, I fixed this problem by adding stub entires for CONFIG_SAMSUNG_PHONE_TTY and CONFIG_SAMSUNG_PHONE_SVNET to Kernel/arch/arm/Kconfig, which allows those options to be preserved by "make defconfig". I've also verified that all the other purged config options aren't used in this source tree.
Attached is a patch containing the modified Kconfig, just apply and clean build as normal. I've also attached two kernel builds for testing, one with the stock initramfs and the other with k0nane's.
Some more notes:
While comparing strings, I noticed that Samsung appears to do some amount of source clean-up and reorganization in preparation of open source packages. For example, there's a bunch of files stored in a "forte" subdirectory for whatever driver that are moved up to the parent, e.g., "Kernel/arch/arm/mach-s5pv210/forte/mach-aries.c" is moved to "Kernel/arch/arm/mach-s5pv210/mach-aries.c". This alone shouldn't be a problem, but if it was done improperly, i.e., not all forte-specific changes were pushed to the parent, some functionality or device-specific bug fixes could be lost.
With regard to my earlier concern about there being a race condition, I'm uncertain at this point if it's actually a problem. It's possible that the missing device definitions are picked up from sysfs by init, and the relevant device nodes created that way. I haven't verified that this is the case, but given that the Epic should be vulnerable to the same condition, but we never observe it, perhaps it's not really an issue. Also, the reason why modules compiled from the source package are much larger than the ones shipped with the initramfs is that they're unstripped. Compiling with the "INSTALL_MOD_STRIP" environment variable defined results in them being stripped upon "make modules_install".
Hope that helps.
mkasick, you are a god. You've done the work that Samsung completely neglected to do - whether deliberately or not.
I suppose this thread can be closed, now. The changes have been pushed to my Github.
Thanks everyone for your input, and mkasick for fixing an obscure problem on a device you don't own!
I implemented a browser based encryption solution which runs on Windows RT (and many other Windows computers). All I wrote was the HTML page, I am leveraging Crypto.JS javascript library for encryption algorithm. I am using the HTML 5 File API implementation which Microsoft provides for reading and writing files.
I make no claim on this but seems to work good for me. Feel free to feedback if you have any suggestions. The crypto.js library supports many different algorithms and configuration so feel free to modify it to your own purposes.
You can download the zip file to your surface, extract it and load the TridentEncode.htm file into Internet Explorer.
If you want to save to custom directory you probably need to load it from the Desktop IE instead of metro IE (to get the file save dialog). I usually drag and drop the file onto desktop IE and from there I can make favorite. This should work in all IE 11 and probably IE 10 browsers... if you use other browsers you may need to copy paste into the fields since the File API implementation seems rather browser specific. Running the html page from the local filesystem means that there is no man-in-the-middle which helps eliminate some of the vulnerabilities of using a javascript crypto implementation. You could also copy the attached zip file to your skydrive to decrypt your files from other computers.
Skydrive files in theory are secure (unless they are shared to public) so this might be useful for adding another layer of protection to certain info.
Again, use at your own risk, but feel free to play around and test it, and offer any suggestions or critiques of its soundness, or just use it as a template for your own apps.
Ok... this is really cool! Nice idea, and a good first implementation.
With that said, I have a few comments (from a security perspective). As an aside, minimized JS is the devil and should be annihilated with extreme prejudice (where not actually being used in a bandwidth-sensitive context). Reviewing this thing took way too long...
1) Your random number generation is extremely weak. Math.random() in JS (or any other language I'm aware of, for that matter) is not suitable for use in cryptographic operations. I recommend reading http://stackoverflow.com/questions/4083204/secure-random-numbers-in-javascript for suggestions. The answer by user ZeroG (bottom one, with three votes, as of this writing) gets my recommendation. Unfortunately, the only really good options require IE11 (or a recent, non-IE browser) so RT8.0 users are SOL.
NOTE: For the particular case in question here (where the only place I can see that random numbers are needed is the salt for the key derivation), a weak PRNG is not a critical failing so long as the attacker does not know, before the attack, what time the function is called at. If they do know, they can pre-compute the likely keys and possibly succeed in a dictionary attack faster than if they were able to generate every key only after accessing the encrypted file.
2) Similarly, I really recommend not using a third-party crypto lib, if possible; window.crypto (or window.msCrypto, for IE11) will provide operations that are both faster and *much* better reviewed. In theory, using a JS library means anybody who wants to can review the code; in practice, the vast majority of people are unqualified to either write or review crypto implementations, and it's very easy for weaknesses to creep in through subtle errors.
3) The default key derivation function (as used for CryptoJS.AES.encrypt({string}, {string})) is a single iteration of MD5 with a 64-bit salt. This is very fast, but that is actually a downside here; an attacker can extremely quickly derive different keys to attempt a dictionary attack (a type of brute-force attack where commonly used passwords are attempted; in practice, people choose fairly predictable passwords so such attacks often succeed quickly). Dictionary attacks can be made vastly more difficult if the key derivation process is made more computationally expensive. While this may not matter so much for large files (where the time to perform the decryption will dominate the total time required for the attack), it could matter very much for small ones. The typical approach here is to use a function such as PBKDF2 (Password-Based Key Derivation Function) with a large number of iterations (in native code, values of 20000-50000 are not uncommon; tune this value to avoid an undesirably long delay) although other "slow" KDFs exist.
4) There's no mechanism in place to determine whether or not the file was tampered with. It is often possible to modify encrypted data, without knowing the exact contents, in such a way that the data decrypts "successfully" but to the wrong output. In some cases, an attacker can even control enough of the output to achieve some goal, such as compromising a program that parses the file. While the use of PKCS7 padding usually makes naïve tampering detectable (because the padding bytes will be incorrect), it is not a safe guarantee. For example, a message of 7 bytes (or 15 or 23 or 31 or any other multiple of 8 + 7) will have only 1 byte of padding; thus there is about a 0.4% (1 / 256) chance that even a random change to the ciphertext will produce a valid padding. To combat this, use an HMAC (Hash-based Message Authentication Code) and verify it before attempting decryption. Without knowing the key, the attacker will be unable to correct the HMAC after modifying the ciphertext. See http://en.wikipedia.org/wiki/HMAC
5) The same problem as 4, but from a different angle: there's no way to be sure that the correct key was entered. In the case of an incorrect key, the plaintext will almost certainly be wrong... but it is possible that the padding byte(s) will be correct anyhow. With a binary file, it may not be possible to distinguish a correct decryption from an incorrect one. The solution (an HMAC) is the same, as the odds of an HMAC collision (especially if a good hash function is used) are infinitesimal.
6) Passwords are relatively weak and often easily guessed. Keyfiles (binary keys generated from cryptographically strong random number generators and stored in a file - possibly on a flashdrive - rather than in your head) are more secure, assuming you can generate them. It is even possible to encrypt the keyfile itself with a password, which is a form of two-factor authentication: to decrypt the data that an attacker wants to get at, they need the keyfile (a thing you have) and its password (a thing you know). Adding support for loading and using keyfiles, and possibly generating them too, would be a good feature.
The solutions to 3-5 will break backward compatibility, and will also break compatibility with the default parameters for openssl's "enc" operation. This is not a bad thing; backward compatibility can be maintained by either keeping the old version around or adding a decrypt-version selector, and openssl's defaults for many things are bad (it is possible, and wise, to override the defaults with more secure options). For forward compatibility, some version metadata could be prepended to the ciphertext (or appended to the file name, perhaps as an additional extension) to allow you to make changes in the future, and allow the encryption software to select the correct algorithms and parameters for a given file automatically.
Wow thanks GDTD that's great feedback
Not sure about his minified sources, the unminified aes.js in components is smaller than the minified version (which I am using) in rollups. I'll have to look into what his process for 'rollup' is to see if I can derive a functional set of non-minified script includes. If I can do that it would be easier to replace (what I would guess is) his reliance on Math.random.
His source here mirrors the unminified files in components folder : https://code.google.com/p/crypto-js/source/browse/tags/3.1.2/src
msCrypto that would be great, I had no idea that was in there. I found a few (Microsoft) samples so I will have to test them out and see if I can completely substitute that for crypto.js. Would be more keeping in line with the name I came up with.
Currently this version only works for text files, I am using the FileAPI method reader.readAsText(). I have been trying to devise a solution for binary files utilizing reader.readAsArrayBuffer but as yet I haven't been able to convert or pass this to crypto.js. I will need to experiment more with base64 or other interim buffer formats (which Crypto.js or msCrypto can work with) until I can get a better understanding of it.
Metadata is a great idea, maybe i can accommodate that with a hex encoded interim format.
You seem extremely knowledgeable in the area of encryption, hopefully i can refine the approach to address some of the issues you raised by setting up proper key, salt, and IV configuration... I'm sure I will understand more of your post as i progress (and after reading it about 20 times more as a reference).
Too bad we don't a web server for RT, that would at least open up localStorage for json serialization (mostly for other apps I had in mind). I guess they might not allow that in app store though. Could probably run one of a developers license though (renewed every 1-2 months)?
nazoraios said:
Too bad we don't a web server for RT, that would at least open up localStorage for json serialization (mostly for other apps I had in mind). I guess they might not allow that in app store though. Could probably run one of a developers license though (renewed every 1-2 months)?
Click to expand...
Click to collapse
I cant comment too much on the encryption, GoodDayToDie has covered anything I could contribute and more. But there is a functioning web server on RT. Apache 2.0 was ported: http://forum.xda-developers.com/showthread.php?t=2408106 I dont know if everything is working on it, I dont own an RT device and last time I tried I couldnt get apache to run on 64 bit windows 8 anyway (needed it at uni, spent hours going through troubleshooting guides and it never worked on my laptop, gave up and ran it under linux in virtualbox where it took 2 minutes to have functioning the way I needed it to).
Curious about the performance. Speaking of encryption, 7-Zip has it built-in, and from the discuss in StackExchange, it seems pretty good.
One of the neat things about this thing (local web app? Pseudo-HTA (HTml Application)? Not sure if there's a proper name for such things) is that it runs just fine even on non-jailbroken devices. That's a significant advantage, at least for now.
Running a web server should be easy enough. I wrote one for WP8 (which has a subset of the allowed APIs for WinRT) and while the app *I* use it in won't be allowed in the store, other developers have taken the HTTP server component (I open-sourced it) and packaged it in other apps which have been allowed just fine. With that said, there are of course already file crypto utilities in the store anyhow... but they're "Modern" apps so you might want to develop such a server anyhow so you can use it from a desktop web browser instead.
Web cryptography (window.crypto / window.msCrypto) is brand new; it's not even close to standardization yet. I'm actually kind of shocked MS implemented it already, even if they put it in a different name. It's pretty great, though; for a long time, things like secure random numbers have required plugins (Flash/Java/Silverlight/whatever). Still, bear in mind that (as it's still far from standardized), the API might change over time.
Yep, I think of them as Trident apps since trident is what Microsoft calls their IE rendering engine, but I guess they are sort of offline web apps (which come from null domain). Being from null domain you are not allowed to use localstorage which is domain specific. You also are not allowed to make ajax requests. You just have file api and json object serialization to make do with I/O.
Another app I am working on is a kind of Fiddler app similar to http://jsfiddle.net/ where you can sandbox some simple script programs.
Kind of turning an RT device into a modern/retro version of a commodore 64 or other on-device development environments. Instead of basic interpreter you've got your html markup and script.
I have an attached demo version which makes available jquery, jquery-ui, alertify javascript libraries in a sandbox environment that you can save as .prg files.
I put a few sample programs in the samples subfolder. Some of the animation samples (like solar system) set up timers which may persist even after cleared so you might need to reload the page to clear those.
It takes a while to extract (lots of little files for all the libraries) but once it extracts you can run the html page and I included a sample program 'Demo Fiddle.prg' you can load and run to get an idea.
I added syntax highlighting editors (EditArea) which seems to work ok and let's you zoom each editor full screen.
The idea would be to take the best third party javascript libraries and make them available and even make shortcuts or minimal API for making it easier to use them. Common global variable, global helper methods, ide manipulation. I'd like to include jqplot for charting graphs, maybe for mathematical programs and provide api for user to do their own I/O within the environment.
These are just rough initial demos, and obviously open source so if anyone wants to take the ideas and run with them i'd be interested in seeing what others do. Otherwise I will slowly evolve the demos and release when there are significant changes.
Hello,
I want to figure out the Samsung Accesory Protocol in order to create a "open source" Gear Manager app replacement. This thread is to ask if anyone has been trying to do the same thing as well as try to gather as much information about this protocol as possible. Generic discussion is also accepted, in case anyone has better ideas.
Right now all I know is that this protocol is based on RFCOMM, albeit it can be transported over TCP too. It has a level 1 "framing" which consists basically on
Code:
packed struct Frame {
uint16_be length_of_data;
char data[length_of_data];
}
packed struct FrameWithCRC {
uint16_be length_of_data;
uint16_be crc_of_length;
char data[length_of_data];
uint16_be crc_of_data;
}
I also know that there are various types of packets. "Hello" packets are exchanged early during the connection and contain the product name, etc. Authentication packets are exchanged right after the initial "hello" and contain some varying hashes (crypto warning!). Then the normal data packets are "multiplexed", as in usbmuxd: they have 'session' IDs which described towards which watch program they are talking with. All Hello and authentication packets are sent without CRC, but normal data packets are. The CRC implementation used is crc16, same poly as in the linux kernel.
I suspect that whatever we uncover about this protocol might be useful to e.g. pair Gear with an iPhone, with a PC, things like that.
Note: most of this comes from viewing Bluetooth logs. However it's clear that reverse engineering will be required for the cryptographic parts. In this case I believe it's legally OK to do so in the EU because it's purely for interoperability reasons. I don't want to create a competitor to the Gear2, I just want to talk to it.
Motivation: I bought a Gear2 in order to replace a LiveView that was dying (buttons wearing out, broken wriststrap clips, etc.) . I used it both for notifications as well as map/navigation.
Since I have a Jolla, no programs are available to pair with most smartwatches, but I've been developing my own so far (MetaWatch, LiveView). Thus I decided on a replacement based purely on hardware characteristics and price. Also Tizen seems more open than Android, thus I figured out it would be easier for me to adapt to the watch.
However it seems that I understimated the complexity of the protocol that connects the Gear with the GearManager. So my options in order to make use of this watch are:
Sell Gear2 back and buy something that's easier to hack (e.g. another LiveView ),
Figure out the SAP protocol and write a replacement Gear Manager app (what this thread is about),
Write replacement Tizen applications that don't use SAP. This involves writing new programs for Calls, Messages, Notifications, Alarms, Camera, watchOn, Pulse monitor, etc. i.e. a _lot_ of work if I want to exploit all features of the watch.
But at least one can reuse the existing Tizen settings app, launcher, drivers, etc. (I started porting Qt to the Gear2 with this idea)
Use a different Linux distro on the Gear 2. Such as Sailfish, Mer, etc. This involves all the work of option 3 + possibly driver work.
As of now I've not decided which option is easier for me so I'll keep trying to push them all.
javispedro said:
Hello,
I want to figure out the Samsung Accesory Protocol in order to create a "open source" Gear Manager app replacement. This thread is to ask if anyone has been trying to do the same thing as well as try to gather as much information about this protocol as possible. Generic discussion is also accepted, in case anyone has better ideas.
Right now all I know is that this protocol is based on RFCOMM, albeit it can be transported over TCP too. It has a level 1 "framing" which consists basically on
Code:
packed struct Frame {
uint16_be length_of_data;
char data[length_of_data];
}
packed struct FrameWithCRC {
uint16_be length_of_data;
uint16_be crc_of_length;
char data[length_of_data];
uint16_be crc_of_data;
}
I also know that there are various types of packets. "Hello" packets are exchanged early during the connection and contain the product name, etc. Authentication packets are exchanged right after the initial "hello" and contain some varying hashes (crypto warning!). Then the normal data packets are "multiplexed", as in usbmuxd: they have 'session' IDs which described towards which watch program they are talking with. All Hello and authentication packets are sent without CRC, but normal data packets are. The CRC implementation used is crc16, same poly as in the linux kernel.
I suspect that whatever we uncover about this protocol might be useful to e.g. pair Gear with an iPhone, with a PC, things like that.
Note: most of this comes from viewing Bluetooth logs. However it's clear that reverse engineering will be required for the cryptographic parts. In this case I believe it's legally OK to do so in the EU because it's purely for interoperability reasons. I don't want to create a competitor to the Gear2, I just want to talk to it.
Motivation: I bought a Gear2 in order to replace a LiveView that was dying (buttons wearing out, broken wriststrap clips, etc.) . I used it both for notifications as well as map/navigation.
Since I have a Jolla, no programs are available to pair with most smartwatches, but I've been developing my own so far (MetaWatch, LiveView). Thus I decided on a replacement based purely on hardware characteristics and price. Also Tizen seems more open than Android, thus I figured out it would be easier for me to adapt to the watch.
However it seems that I understimated the complexity of the protocol that connects the Gear with the GearManager. So my options in order to make use of this watch are:
Sell Gear2 back and buy something that's easier to hack (e.g. another LiveView ),
Figure out the SAP protocol and write a replacement Gear Manager app (what this thread is about),
Write replacement Tizen applications that don't use SAP. This involves writing new programs for Calls, Messages, Notifications, Alarms, Camera, watchOn, Pulse monitor, etc. i.e. a _lot_ of work if I want to exploit all features of the watch.
But at least one can reuse the existing Tizen settings app, launcher, drivers, etc. (I started porting Qt to the Gear2 with this idea)
Use a different Linux distro on the Gear 2. Such as Sailfish, Mer, etc. This involves all the work of option 3 + possibly driver work.
As of now I've not decided which option is easier for me so I'll keep trying to push them all.
Click to expand...
Click to collapse
I think your thread should probably go in the Dev section for Tizen. Have you made any development? If your want it moved, report your own post with the button in top right labeled report. You can then suggest your thread be moved to the new Tizen Development section. Ok, I wish you all the luck, you seem to be very talented programmer/dev. Thanks for your contributions.
Chris
noellenchris said:
I think your thread should probably go in the Dev section for Tizen.
Click to expand...
Click to collapse
Well, some mod already moved this thread from Development, where I originally posted it, into Q&A. This is not exactly "Tizen" development (SAP is used in may Samsung devices seemingly).
noellenchris said:
Have you made any development?
Click to expand...
Click to collapse
Yes, lots of progress. I have been able to write a program that connects to the Gear2 from my PC, succesfully "completes" the setup program and synchronizes the date&time. Things like changing the background color etc. are now trivial. I will soon port it to my Jolla.
I am now looking into how to send notifications to the watch. I've not been able to get Gear Manager to actually send any notifications (to use as "reference"), because goproviders crashes when I try to simulate notifications on my android_x86 VM
If anyone can send me an HCI / Bluetooth packet capture of their Android device while it is sending notifications to the Gear2 I would really appreciate it.
Unfortunately, the main problem here is that Samsung uses some cryptographic authentication as a form of "DRM". I am not exactly sure why.
There was no way for me to discover how the crypto worked so I took the unclean approach and dissasembled their crypto code (libwms.so). That means there's no way I would be able to distribute the code now without risking a lawsuit from Samsung.
Sadly this means that while I can distribute the protocol specifications I obtained, legally distributing "Gear Manager replacements" is probably impossible.
javispedro said:
Well, some mod already moved this thread from Development, where I originally posted it, into Q&A. This is not exactly "Tizen" development (SAP is used in may Samsung devices seemingly).
Click to expand...
Click to collapse
Ya, I was kinda in a Gear 1 mind set, and they have separate threads for Android and Tizen....
Chris
javispedro said:
Unfortunately, the main problem here is that Samsung uses some cryptographic authentication as a form of "DRM". I am not exactly sure why.
There was no way for me to discover how the crypto worked so I took the unclean approach and dissasembled their crypto code (libwms.so). That means there's no way I would be able to distribute the code now without risking a lawsuit from Samsung.
Sadly this means that while I can distribute the protocol specifications I obtained, legally distributing "Gear Manager replacements" is probably impossible.
Click to expand...
Click to collapse
I would gladly write a MIT-licensed C library implementing your protocol specifications. That would be correctly following the chinese-wall approach to reverse-engineering, right?
Anyway, AFAIK, being in Europe decompiling for interoperability purposes is allowed -- I know that wikipedia is not to be taken at face value, but: en.wikipedia.org/wiki/Reverse_engineering#European_Union
Antartica said:
I would gladly write a MIT-licensed C library implementing your protocol specifications. That would be correctly following the chinese-wall approach to reverse-engineering, right?
Anyway, AFAIK, being in Europe decompiling for interoperability purposes is allowed -- I know that wikipedia is not to be taken at face value, but: en.wikipedia.org/wiki/Reverse_engineering#European_Union
Click to expand...
Click to collapse
Well, the problem is not the protocol specifications per se, which I'm actually quite confident I'd be able to redistribute (I'm in EU). The problem is the cryptography part, which is basically ripped off from the Samsung lib "libwsm.so" . Unless we can find out what cryptographic method that lib uses, distributing alternate implementations Is a no-go.
javispedro said:
Well, the problem is not the protocol specifications per se, which I'm actually quite confident I'd be able to redistribute (I'm in EU). The problem is the cryptography part, which is basically ripped off from the Samsung lib "libwsm.so" . Unless we can find out what cryptographic method that lib uses, distributing alternate implementations Is a no-go.
Click to expand...
Click to collapse
If you have the time, I don't mind researching the possible crypto used (although I've only studied DES/3DES, AES and Serpent, hope that whatever scheme used is not very different from them).
Some ideas to start from somewhere:
1. As you have used its functions, it is a block cipher? I will assume that it is.
2. What is the key size and the block size?
3. Are there signs that it is using a stack of ciphers? (that is, applying one cipher, then another to the first result and so on)
Antartica said:
If you have the time, I don't mind researching the possible crypto used (although I've only studied DES/3DES, AES and Serpent, hope that whatever scheme used is not very different from them).
Some ideas to start from somewhere:
1. As you have used its functions, it is a block cipher? I will assume that it is.
2. What is the key size and the block size?
3. Are there signs that it is using a stack of ciphers? (that is, applying one cipher, then another to the first result and so on)
Click to expand...
Click to collapse
Hello, I've not forgotten about this, just somewhat busy and been using the MetaWatch lately
1. Yes it is clearly a block cipher, and the block size Is 16bytes.
2. I don't know about the key size, it is obfuscated.
3. Doesn't seem like a stack of ciphers. It looks like some overcomplicated AES. But to be honest AES is the only encryption I know of
By the way I think I will upload my current test "manager" source code to somewhere after removing the crypto specific files . Since the protocol itself has been obtained cleanly. Note I've used Qt (not the GUI parts) so it's useless for creating a library; the code will probably need to be rewritten to do so, but it may be useful as "protocol specs".
javispedro said:
Hello, I've not forgotten about this, just somewhat busy and been using the MetaWatch lately
Click to expand...
Click to collapse
No problem. Curiously, I've transitioned from the metawatch to the Gear1 fully (null rom, not pairing with bluetooth to the phone but gear used as a standalone device).
[off-topic]I'm not using my metawatch anymore. I was modifying Nils' oswald firmware to make it prettier and to have some features I wanted (calendar, stopwatch), but it was very inaccurate, supposedly because of missing timer interrupts (the existing LCD drawing routines were too slow). I rewrote the graphics subsystem just to stumble into a known mspgcc bug, and trying to use the new redhat's mspgcc resulted in more problems (memory model, interrupt conventions). In the end I couldn't commit enough time to fix that and my metawatch is now in a drawer[/off-topic]
Returning to the topic:
javispedro said:
1. Yes it is clearly a block cipher, and the block size Is 16bytes.
Click to expand...
Click to collapse
Good. We can at least say it isn't DES/3DES nor blowfish (64 bits block size). Regrettably there are a lot of ciphers using 128-bits block size; that I know: AES, Twofish and serpent.
Perusing the wikipedia there are some more of that size in use: Camellia, sometimes RC5 and SEED.
javispedro said:
2. I don't know about the key size, it is obfuscated.
3. Doesn't seem like a stack of ciphers. It looks like some overcomplicated AES. But to be honest AES is the only encryption I know of
Click to expand...
Click to collapse
I understand that to mean that you cannot use that library passing your own key, right?
What a pity! One way to test for these ciphers would have been to just cipher a known string (i.e. all zeroes) with a known key (i.e. also all zeroes) and compare the result with each of the normal ciphers :-/.
javispedro said:
By the way I think I will upload my current test "manager" source code to somewhere after removing the crypto specific files . Since the protocol itself has been obtained cleanly. Note I've used Qt (not the GUI parts) so it's useless for creating a library; the code will probably need to be rewritten to do so, but it may be useful as "protocol specs".
Click to expand...
Click to collapse
Perfect. I don't need anything more .
Ok, so I've uploaded my SAP protocol implementation: https://git.javispedro.com/cgit/sapd.git/ . It's "phone" side only, ie it can be used to initiate a connection to the watch but not to simulate one. In addition, it's missing two important files: wmscrypt.cc and wmspeer.cc which implement the closed crypto required to "pair" the watch. The most important file is sapprotocol.cc which implements the packing/unpacking of the most important packet types. The license of those files is GPLv3 albeit I'm very happy if you use the information contained on them to build your "Gear Manager" program under whichever license you'd prefer.
For anyone who hasn't been following the above discussion: I've figured out a large part (useful for at least establish contact with the watch and syncing time/date) of the SAP protocol used between the Gear watch and the Gear manager program on the phone. This has been done mostly by studying traces and afterwards talking to the watch using my test implementation above to figure out the remaining and some error codes. The debug messages left by the watch's SAP daemon were also immensely helpful. As long as I understand this is perfectly safe to do, publish and use as I'm in the EU and is basically the same method Samba uses.
Unfortunately, the protocol contains some crypto parts required for the initial sync (subsequent connections require authentication). However, the communication itself is not encrypted in any way, which helped a lot with the process. Because it's impossible for me to figure out whatever authentication method is used, I had to disassemble the library implementing this stuff (libwms.so). This is still OK according to EU law, but I'm no longer to release that information to the public. I'm looking for alternatives or ideas on how to handle this fact.
In the meanwhile, let's talk about the protocol. It's basically a reimplementation of the TCP(/IP) ideas on top of a Bluetooth RFCOMM socket. This means that it's connection oriented and that it can multiplex several active connections (called "sessions") over a single RFCOMM link. Either side of the connection can request opening a connection based on the identifier of the listening endpoint (called a "service"). Strings are used to identify services instead of numeric ports as in TCP. For example, "/system/hostmanager" is a service that listens on the watch side. Once you open a session towards this service (i.e. once you connect to it) you can send the time/date sync commands. In addition to be the above the protocol also seems to implement QoS and reliability (automatic retransmission, ordering, etc.). It's not clear to me why they reimplemented all of this since RFCOMM is a STREAM protocol, and thus reliability is already guaranteed!! So I've not focused much on these (seemingly useless) QoS+reliability parts of the protocol.
Let's start with the link level. There are two important RFCOMM services exposed by the watch: {a49eb41e-cb06-495c-9f4f-aa80a90cdf4a} and {a49eb41e-cb06-495c-9f4f-bb80a90cdf00}. I am going to respectively call those two services "data" and "nudge" from now on. These names, as many of the following ones, are mostly made up by me .
The communication starts with Gear manager trying to open a RFCOMM socket towards the "nudge" service in the watch. This causes the watch to immediately reply back by trying to open a connection to the "data" service _on the phone_ side. So obviously this means that your phone needs to expose the "data" RFCOMM service at least. In addition, the watch will try to open a HFP-AG connection (aka it will try to simulate being a headset) to your phone. Most phones have no problem doing this so no work is required. Of course, if your phone is a PC (as in my case ) then you'll need to fake the HFP profile. I give some examples in my code above (see scripts/test-hfp-ag and hfpag.cc).
Once the RFCOMM socket from the watch to the phone "data" service is opened, the watch will immediately send what I call a "peer description" frame. This includes stuff such as the model of the watch as well as some QoS parameters which I still don't understand. The phone is supposed to reply back to this message with a peer description of its own. See sapprotocol.cc for the packet format.
After the description exchange is done, the watch will send a "authentication request" packet. This is a 65 byte bigint plus a 2 byte "challenge". The response from the phone should contain a similar 65 byte bigint, the 2 byte response, and an additional 32 byte bigint. If correct, the watch will reply with some packet I don't care about. Otherwise the connection will be dropped. It obviously looks like some key exchange. But this is the crypto part that's implemented in libwms.so....
After these two exchanges link is now set up. The first connection that needs to be opened is towards a service that is always guaranteed to be present, called "/System/Reserved/ServiceCapabilityDiscovery". It is used by both sides of the connection to know the list of available services present on the other side. Despite this, you cannot query for all services; instead, you must always know the name of the remote service you're looking for. There's some 16-byte checksum there which I don't know how to calculate, but fortunately the watch seems to ignore it!! I suspect that you're expected to actually persist the database of available services in order to shave a roundtrip when connection is being established. But this is not necessary for normal function. This service is implemented in capabilityagent.cc, capabilitypeer.cc . This part was actually one of the most complex ones because of the many concepts. I suggest reading the SDK documentation to understand all the terms ("service", "profile", "role", etc.).
If everything's gone well, now the watch will try to open a connection to a service in your phone called "/system/hostmanager". Once you get to this message things start to get fun, because the protocol used for this service is JSON! It's implementation resides in hostmanageragent.cc, hostmanagerconn.cc . For example, Gear Manager sends the following JSON message once you accept the EULA: {"btMac":"XX:XX:XX:XX:XX:XX", "msgId":"mgr_setupwizard_eula_finished_req", "isOld":1}. At this point, the watch hides the setup screen and goes straight to the menu.
Well, this concludes my high-level overview of the SAP protocol. Hope it is useful for at least someone!
Things to do:
Personally I'm looking for some traces of the notification service. Ie the one that forwards Android notifications towards the watch. For some reason it doesn't work on my phone, so I can't get traces. I suspect it's going to be a simple protocol so a few traces will be OK. It's the only stuff I'm missing in order to be able to actually use the Gear as a proper smartwatch with my Jolla.
We still need to tackle the problem of the cryptographic parts. Several options: either "wrap" the stock libwms.so file, try to RE it the "proper way", .... I'm not sure of the feasibility of any of these.
Many other services.
javispedro said:
After the description exchange is done, the watch will send a "authentication request" packet. This is a 65 byte bigint plus a 2 byte "challenge". The response from the phone should contain a similar 65 byte bigint, the 2 byte response, and an additional 32 byte bigint. If correct, the watch will reply with some packet I don't care about. Otherwise the connection will be dropped. It obviously looks like some key exchange. But this is the crypto part that's implemented in libwms.so....
Click to expand...
Click to collapse
About that 65-byte bigint... that is a 520-bit key. The usual length of ECDSA keys is exactly 520-bits, so we may have something there: it is possible that they are using ECDSA signing (just like in bitcoin, so there are a lot of implementations of that code).
Not forgotten about this!
Just an status update:
I'm still in the process of defining the API of the C library using javispedro's sources as template.
It's tougher than I originally supposed because the C++ code has a lot of forward-declarations of classes, which is very difficult to map into C. To counter that I have to move elements between structures and I'm not so comfortable with the codebase yet.
And then there is still the hard work of translating the Qt signals/slots to plain' old callbacks... and implementing the bluetooth part using bluez API... and... well, I hope that is all.
Anyway, patience .
I've now had access to a Samsung S2 and thus I have been able to obtain more traces. The latest Git now contains code to connect to the notification manager service, thus allowing to send notifications from the phone to the watch.
That was the last missing part to be able to use the Gear 2 as a 'daily' smartwatch with my Jolla, so I've now also ported the code to run under Sailfish. In fact I'm using this setup at the moment. My first comment is "wow the vibrator IS weak".
You can find a log of sapd's (ie my code) startup qDebug() messages; they may be useful (if you can't yet get your code to run)
I suspect that there may still be some important battery issues because the watch keeps printing error messages about SAP services it can't find on the phone (and instead of sleeping, it starts busy polling for them.... :/ ). It does not seem to happen while the watch is out of the charging cradle, so it may not be important, but not sure yet.
As for the encryption, I'm not sure how to proceed. I could describe the code to you, but that would be risky, because I don't understand what it does. Thus the only way (for me) to describe it would be to pass on the mathematical formulas/pseudocode ... Apart from that, we also have the problem of the keys...
Antartica said:
The usual length of ECDSA keys is exactly 520-bits, so we may have something there: it is possible that they are using ECDSA signing
Click to expand...
Click to collapse
They do use ECDH indeed, and they link with OpenSSL and import the ECDH functions. However it's not clear if they use ECDSA; while the crypto algorithm DOES resemble DSA, I cannot fully identify it.
Congratulations for managing to make it work with the Jolla .
I have finally found a suitable "flattened" class hierarchy as to be able to map your code into C; see the attachs. Basically, I have to move the functionality of SAPConnectionRequest, SAPSocket, CapabilityPeer and SAPConnection into SAPPeer, and then it is suitable for my needs.
javispedro said:
As for the encryption, I'm not sure how to proceed. I could describe the code to you, but that would be risky, because I don't understand what it does. Thus the only way (for me) to describe it would be to pass on the mathematical formulas/pseudocode ... Apart from that, we also have the problem of the keys...
They do use ECDH indeed, and they link with OpenSSL and import the ECDH functions. However it's not clear if they use ECDSA; while the crypto algorithm DOES resemble DSA, I cannot fully identify it.
Click to expand...
Click to collapse
If you manage to describe it using mathematical formulas as in
http://en.wikipedia.org/wiki/Ellipt...ture_Algorithm#Signature_generation_algorithm
it would be perfect, but I reckon that to be able write that you need intimate knowledge of the code and don't know if you have time for that :angel:
And identifying the hash function used would be a problem in itself...
One idea: how about a ltrace so we have the calls to the openssl library? That may uncover new hints.
Anyway, I have a lot of work before me until I need that, so don't fret over it.
Hi there! Any chance that the Gear can (really) work with an iPhone?
gidi said:
Hi there! Any chance that the Gear can (really) work with an iPhone?
Click to expand...
Click to collapse
agreed. Needs iPhone support please.
Antartica said:
Congratulations for managing to make it work with the Jolla .
I have finally found a suitable "flattened" class hierarchy as to be able to map your code into C; see the attachs. Basically, I have to move the functionality of SAPConnectionRequest, SAPSocket, CapabilityPeer and SAPConnection into SAPPeer, and then it is suitable for my needs.
Click to expand...
Click to collapse
You may want to look at the official Samsung SDK docs to match their class hierarchy. I tried to match my hierarchy to theirs, but this happened very late in the development process, so there is some weirdness.
Antartica said:
One idea: how about a ltrace so we have the calls to the openssl library? That may uncover new hints.
Click to expand...
Click to collapse
I more or less know what it is doing with OpenSSL, but that's because I looked at the dissassembly. They use OpenSSL for key derivation (ECDH), but the actual cryptographic algorithm is their own. This 'block cipher' is the part they have tried to obfuscate. Not much, but still enough to require more time than what I have available It is basically a set of arithmetical operations with some tables hardcoded in the libwsm.so binary, so no external calls to any library. The hardcoded tables are probably derivated from their private key, which is most definitely not on the binary. In fact I suspect this is basically AES with some changes to make it hard to extract the actual key used, so that's where I've centered my efforts.
Technically it should not even be copyrightable, so maybe I could just redistribute my C reimplementation of the algorithm, but as with any other DRM who knows these days... and that still leaves the problem of the tables/"private key".
Digiguest said:
agreed. Needs iPhone support please.
Click to expand...
Click to collapse
Well you are welcome to implement one such iPhone program yourself. Will be happy to resolve all the protocol questions you have.
(But please stop with the nagging).
Wasn't nagging at all. Just agreeing with him. I am no programmer so I have to rely on others for answers. Sorry if you thought otherwise.
Looking for to see more work on it though. Keep it up.
Hi there! Nice work on getting Gear2 to work with Jolla.
I'd love to get Gear1 to work with WP8.1. Do you have the code for Jolla
on github/bitbucket so I could give it a peek? Thanks in advance.
Duobix said:
Hi there! Nice work on getting Gear2 to work with Jolla.
I'd love to get Gear1 to work with WP8.1. Do you have the code for Jolla
on github/bitbucket so I could give it a peek? Thanks in advance.
Click to expand...
Click to collapse
javispedro had the sources in gitorius, but they are not there anymore (surely related to gitlab buying gitorius).
I attach a tarball with javispedro sources as of 19 October 2014.
Note that it lacks the files implementing the crypto, so just porting it is not enough to be able to communicate to the gear. OTOH, I know that there are some differences in the protocol between the Android Gear1 and the Tizen Gear2 (if the gear1 has been updated to Tizen, it uses the same protocol as gear2). Specifically, to be able to communicate with both watches, the gear manager package has both gear manager 1.7.x and gear manager 2.x. javispedro's code implements the gear 2 protocol.
Personally, I have my port on hold (I have problems with bluetooth in my phone, so there is no point in porting sapd right now as I would not be able to use it).