Related
I have posted a device tree for Gingerbread for the HeroC. This has not had a lot of mileage, so don't expect that it will be at the level of AOSPs builds, but is a starting point for those who would like to build their own GB Roms.
Credit for this should go to those who did the REAL work, such as AOSP, Lox, Elemag, Darchstar, Cyanogen, and a host of others.
My contribution was just the assembly and cleanup of the existing efforts of others.
Still plenty of issues that need working on, and not sure how much time I will have to devote to moving this forward. I have tried to make it fairly self contained, with enough information to get folks started. I will not be able to provide much support, but I am making it available in the event that it is of interest.
Special thanks to AOSP for a working ROM to look at and learn from, and appreciation for some files and info that I shamelessly kanged from his build. Also thanks to Lox and Elemag for their hero device tree which also provided some significant pieces of the puzzle.
Good luck, thanks for the community, and please continue to help each other.
You can get the device tree from github by:
git clone git://github.com/wjb/android_device_htc_heroc_gb.git
README
Basic steps to build:
1 - Get local copy of CM7 repository
2 - Get copy of the device tree into device/htc/heroc
3 - Copy modified files from 'device/htc/heroc/buildmods' to proper directories:
a) definitions.mk into 'build/core' (Temporary fix for boot.img size issue)
b) vendorsetup.sh into 'vendor/cyanogen' (Add lunch combo)
c) prelink-linux-arm-heroc.map into 'vendor/cyanogen' (Map library locations)
d) AndroidProducts.mk into 'vendor/cyanogen/products' (Add cyanogen_heroc product)
e) cyanogen_heroc.mk into 'vendor/cyanogen/products' (Add cyanogen_heroc makefile)4 - Run get-rommanager in vendor/cyanogen directory
5 - Get proprietary files into 'vendor/htc/heroc/proprietary'
a) Put ROM (update.zip) in repo root and run 'unzip-files.sh' from the 'device/htc/heroc' directory
b) Use 'adb' to pull files from usb connected device by running 'extract-files.sh'
c) Copy files from expanded system directory on your local machine by running 'copy-files.sh'
d) Copy files from 'device/htc/heroc/proprietary' directory (also needs 'setup-makefiles.sh')6 - Setup build environment by sourcing envsetup file '. build/envsetup.sh'
7 - Run 'lunch' to select the build target 'lunch cyanogen_heroc-eng'
8 - Build the system with make, mka, etc.
Save for Updates
gingerbread futures
downloading does it compile witch working cam sensors etc.
aaronpoweruser said:
downloading does it compile witch working cam sensors etc.
Click to expand...
Click to collapse
It does compile and build a working ROM, including camera, gps, compass, wifi, etc.
If you have built other ROMS, this should be very straightforward. If not, follow standard guides for setting up and configuring your build environment.
Whoa jaybob, you're working on GB now?
Oh hell yes. Your updating of themes made CM6 the Goliath(sp?) I loved.
Welcome from the coldness that is frozen yogurt, and into the warmth of gingerbread.
Wait, let me clarify here, jaybob, are you going to put out new builds of CM7 as it comes out, in the same way you were before? I found yours to be of far higher quality than the nightly/stable releases.
Repo Update
Updated the repo to eliminate conflicts with 'lunch' for makefiles included in the buildmods directory.
Also added the quick and dirty 'install-buildmods.sh' script to copy the files from buildmods to the appropriate directories. Executing this script from the buildmods directory takes care of the copy actions indicated in step 3.
LiquidSolstice said:
Wait, let me clarify here, jaybob, are you going to put out new builds of CM7 as it comes out, in the same way you were before? I found yours to be of far higher quality than the nightly/stable releases.
Click to expand...
Click to collapse
ATM, I am just working on getting an environment that will let me build from source as it continues to develop. Without a device tree that works, there was not much I could do.
As things progress, I may put out CM7 builds, but I am still a ways from that point. I am starting to look at the TMO Theme Chooser approach to building installable themes. This would produce themes that are independent of the ROM, which would probably end up being a lot easier than the 5 separate builds that I produced for CM6.
I have only been playing with this for a couple days so far. Fortunately, there was a lot of other development efforts to leverage. I will continue to move this forward as my time and interest allows.
jaybob413 said:
ATM, I am just working on getting an environment that will let me build from source as it continues to develop. Without a device tree that works, there was not much I could do.
As things progress, I may put out CM7 builds, but I am still a ways from that point. I am starting to look at the TMO Theme Chooser approach to building installable themes. This would produce themes that are independent of the ROM, which would probably end up being a lot easier than the 5 separate builds that I produced for CM6.
I have only been playing with this for a couple days so far. Fortunately, there was a lot of other development efforts to leverage. I will continue to move this forward as my time and interest allows.
Click to expand...
Click to collapse
Awesome, really liked your past work.
jaybob413 said:
ATM, I am just working on getting an environment that will let me build from source as it continues to develop. Without a device tree that works, there was not much I could do.
As things progress, I may put out CM7 builds, but I am still a ways from that point. I am starting to look at the TMO Theme Chooser approach to building installable themes. This would produce themes that are independent of the ROM, which would probably end up being a lot easier than the 5 separate builds that I produced for CM6.
I have only been playing with this for a couple days so far. Fortunately, there was a lot of other development efforts to leverage. I will continue to move this forward as my time and interest allows.
Click to expand...
Click to collapse
Take your time. The work you do is quality stuff.
Thanks for this! Worked great, once I actually read the "also needs 'setup-makefiles.sh'" part and ran it. For whatever reason Gallery3D didn't get built, but I just downloaded it from the market. Everything else works as expected!
Worked great! Thanks for your efforts.
Sent from my HERO200 using XDA App
Repo Update
Updated kernel, wlan.ko, and additional loadable kernel modules after kernel rebuild to add scaling_available_frequencies virtual file needed by CPU settings.
Added libspeech.so to proprietaries, and minor build config changes.
The Kernel Source Saga
aka
How Samsung Is Continuing to Violate the GPL
As you are all aware, there are still no custom kernels or any functioning custom recoveries for this fine (despite its flaws) device - the Samsung Galaxy Indulge (specifically, the SCH-R910). We'd love to be able to say that's not the case, but it is - and will continue to be, for as long as the issues outlined below continue. Samsung has left us not just in the lurch, but completely screwed - whether it is simple incompetence, disregard for the prepaid device community, interference by MetroPCS, or flat-out malice, the outcome is the same.
The Symptoms
The problem here is not that Samsung has not published some sort of source package. They have done at least that - enough to keep most observers happy. However, this package was broken from the start. The first package that Samsung published for the R910 included a badly-broken netfilter, which stumped our kernel experts for a good while before the decision was made to simply substitute the three pieces of netfilter from another device.
Doing this allowed our kernel developers (and myself) to compile a kernel, linked with a clean initramfs extracted from the stock kernel. In a properly-released source package, this should be enough - but it wasn't. These kernel attempts would appear to boot, but about 10-15s after completion of the boot animation, would die completely. The LTE and CDMA baseband versions, in About Phone, would display as Unknown, if one could get there quickly enough.
Three of the top Epic kernel developers, myself, and a long-time Android veteran analyzed kmsg logs (see below) and other factors, and determined that it was impossible to generate what is in the stock binary blob from the source package published by Samsung.
Warning: technical details! The following IRC log summarizes the issue:
[10:29] <@k0nane> The kernel now builds successfully - no more errors in netfilter, out of the box - but with the stock initramfs, it will exhibit the same behavior as previous builds. After boot, it will shut down within 10-15s. If there is time to get to About Phone, it will show the baseband versions as CDMA: unknown and LTE: unknown.
[10:30] <@k0nane> The modules that are built, which is not the complete set as in the stock initramfs, are /significantly/ larger than the stock copies and cannot be put in a kernel build as even with LZMA compression it will be far too large to fit in bml8.
[10:31] <@k0nane> So as before, if the solution is to use the (again, incomplete) newly-compiled modules, it's no solution at all. Thus there is still no way to build a kernel identical to the stock build.
[see logs below]
[10:36] <@k0nane> Expert analysis caught one of the anomalies:
[10:36] <@k0nane> <@Decad3nce> DRockstar: <4>[ 6.042947] [MULTIPDP][poll_thread] dpram_open failed!!
[10:36] <@k0nane> <@Decad3nce> probe never goes through
[10:36] <@k0nane> <@Decad3nce> and you fails
[10:36] <@k0nane> <@Decad3nce> :x
[10:36] <@k0nane> <@Decad3nce> what should happen (from start)
[10:36] <@k0nane> <@Decad3nce> <4>[ 6.275749] [MULTIPDP] dpram_open successd !!!!
[10:36] <@k0nane> <@Decad3nce> <3>[ 6.275770] [IDPRAM] <dpram_tty_ioctl:1718> IOCTL cmd = 0x5405
[10:36] <@k0nane> <@Decad3nce> <3>[ 6.280198] [IDPRAM] <dpram_tty_ioctl:1718> IOCTL cmd = 0x540
[10:36] <@k0nane> <@Decad3nce> which triggers phone active
[10:36] <@k0nane> <@Decad3nce> <3>[ 6.757528] [IDPRAM] <phone_active_irq_handler:2195> PHONE_ACTIVE level: LOW , phone_sync: 0
[10:36] <@k0nane> <@Decad3nce> then you have your power on, etc
[10:36] <@k0nane> <@Decad3nce> 3>[ 9.870551] [IDPRAM] dpram_phone_power_on() ...
[10:36] <@k0nane> <@Decad3nce> <3>[ 10.450069] [IDPRAM] <phone_active_irq_handler:2195> PHONE_ACTIVE level: LOW , phone_sync: 0
[10:36] <@k0nane> <@Decad3nce> <3>[ 10.457065] [IDPRAM] <phone_active_irq_handler:2195> PHONE_ACTIVE level: HIGH, phone_sync: 0
[10:36] <@k0nane> <@Decad3nce> also, not sure what this is
[10:36] <@k0nane> <@Decad3nce> <4>[ 6.042931] [MULTIPDP]filp_open() failed~!: -6
[10:36] <@k0nane> <@Decad3nce> doesn't occur in your goodlog
[10:36] <@k0nane> <@DRockstar> hmm, what could prevent the probe?
[10:36] <@k0nane> <@Decad3nce> the dpram_open failing
[10:37] <@k0nane> And to summarize that, "so from boot, goes through kernel init, then device init, then when it gets to the radio interface layer.. it dies a horrible death because of no access to dpram0"
[10:37] <@k0nane> The details should be unimportant, however, if what's provided is identical to/can produce stock (which, as we see, it cannot).
[10:39] <@k0nane> http://k0nane.info/size.png <== a comparison of stock (left) vs. compiled (right) module sizes
Click to expand...
Click to collapse
Logs: Bad Boot #1 Bad Boot #2 Clean Boot
Source and initramfs as of these logs: Github
What Has Samsung Done?
I have been in contact with SamsungJohn (John Imah), Samsung's official developer representative on Twitter. On first notification of the issue, no action had been taken, and it took an RT campaign to catch Mr. Imah's eye. Approximately one week later, Mr. Imah joined me in IRC, and took note of the information above. Shortly after that, Samsung released a new source package on opensource.samsung.com. That was great - except that the only difference was an end to the out-of-the-box compile errors. Everything else about this source package was identical. The results were the same.
Within the next business day, Mr. Imah requested that I send the information to his corporate email address, as he had an "HQ team" there with him. Unfortunately, that was June 22. Mr. Imah has not responded to tweets or DMs since then, and we have been given absolutely no indication that any work has been done on this issue. The source package has not been updated again.
I have given Samsung ample notification and time to work with myself and my fellow developers to resolve this issue without a public expose. They have failed to so much as maintain the appearance of progress, let alone make any. It was, is, time for the saga to end. Samsung is hereby on notice. They, by failing to provide the source necessary to generate what is included in the stock kernel binary, are violating the General Public License.
http://www.gnu.org/copyleft/gpl.html said:
You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways:
- a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange.
- b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge.
- c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b.
- d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.
- e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d.
Click to expand...
Click to collapse
How Can You Help?
Tweet, email, and otherwise contact every public-facing Samsung representative.
Tell your friends. Have them do the same.
Send a copy of this report to [email protected] - the source in question is the Linux kernel, which is owned by the Free Software Foundation.
Just spread the word - until Samsung feels the heat from this (feel free to contact their legal department!) we will not see results. Development for the Indulge will continue to be held back. Only the users will suffer.
--
Follow me on Twitter @k0nane/@publik0.
x-posted from AndroidForums.
for anyone that thinks this isnt worthwhile, imagine if your device had no source.. where would your kernels be? imagine if this became the trend, to give out crap source
shabbypenguin said:
for anyone that thinks this isnt worthwhile, imagine if your device had no source.. where would your kernels be? imagine if this became the trend, to give out crap source
Click to expand...
Click to collapse
And on top of that, I have every reason to believe - though IANAL - their behavior is illegal.
RT the thread.
I blew this up over on samsungs facebook page.. someone got pissed and deleted my post..
http://www.facebook.com/SamsungMobileUSA
I started to 'comment' on a bunch of their stuff w/ a link to this thread..
I might get my account suspended BWAHAHAHAHA..
bastards
Let's call out @samsungjohn on twitter!
Sent from my Epic™ 4G using XDA Premium app
This is the vary reason i will NEVER BUY any thing sumsuck ever makes, EVER AGAIN!
I have a suspicion that what's going on here is actually just a (possibly long-standing) bug. However, the situation is a bit strange all around.
To be specific, I suspect it's a race condition that happens to be triggered by your kernel build, but not Samsung's. This could easily be due to nothing more than using a different compiler, i.e., one that produces more efficient code.
I notice that in "Bad Boot #1" and "Bad Boot #2", the "[MULTIPDP][poll_thread] dpram_open failed!!" message is preceeded by "[MULTIPDP]filp_open() failed~!: -6". Looking at multipdp.c, dpram_open is failing to open "/dev/dpram1" and returning ENXIO ("No such device or address"), indicating that /dev/dpram1 doesn't exist yet.
Now, there's only two references to "/dev/dpram1" in the entire stock kernel image. The first is the line in multipdp.c that it's failing on. The second is in initramfs's /sbin/init. Here, Android's init actually contains a simplified implementation of udev to handle kernel uevents. That is, it's responsible for dynamically creating device nodes as kernel drivers are loaded, one of which is /dev/dpram1.
What I think is happening is this:
init.rc insmods dpram.ko
dpram.ko registers the dpram tty devices, issuing an add uevent message for device 252:2.
... (some time elapses) ...
init processes the uevent message, creates /dev/dpram1.
Meanwhile:
init.rc insmods multipdp.ko
multipdp.ko creates the poll_thread kernel thread.
poll_tread calls dpram_open.
dpram_open attempts to open /dev/dpram1, which might not exist yet.
Probably what's happening is that Samsung's kernel "takes long enough" to get to the "insmod multipdp.ko" part of init.rc that init has time to process the dpram.ko uevents and create /dev/dpram1, so that by the time dpram_open is called, the device node exists. However, your kernel is "fast enough" that init.rc gets to "insmod multipdp.ko", creates the new kernel thread and executes it, before init has caught up on device node creation. Thus, /dev/dpram1 doesn't exist and everything goes to ****.
Now here is where it gets bizarre. If we compare against the Epic's EC05 init.rc, there's an interesting stanza prior to module initialization that's missing here:
Code:
#samsung dpram modules
mknod 0666 /dev/dpram0 c 255 1
mknod 0666 /dev/dpram1 c 255 2
mknod 0666 /dev/dpramerr c 255 0
mknod 0666 /dev/ttyCDMA0 c 240 1
Presumably this is supposed to create the missing device nodes before insmoding both dpram.ko and multipdp.ko, eliminating the race condition. However, mknod is a command not supported by init, there's no mknod program in the initramfs, and those are the wrong device numbers. Perhaps it's an acknowledgment that the race condition exists on both devices, even though nothing is actually done on the Epic to avoid it? It's just, ... strange.
Unfortunately I can't think of a great solution. One option would be to modify multipdp.c to add a sleep loop around "filp_open(DPRAM_DEVNAME, ...)" to wait for the device to be created. However, that would mean using your own compile of multipdp.ko instead of the one Samsung provides, which could open a different can of worms.
A second option is to move "insmod /lib/modules/multipdp.ko" until after mounting /system, hoping that init would've caught up by then. This can even be insured by running a script that checks for the device node and sleeps if it doesn't exist yet. Something like:
Code:
#!/system/bin/sh
for i in 1 2 3 4 5; do
ls /dev/dpram1 && break
sleep 1
done
and call it from init.rc, after mounting /system, with:
Code:
exec /system/etc/wait_for_dpram1.sh
Anyways, even if this particular issue turns out not to be the culprit, I wouldn't be too quick to conclude that Samsung must've released bad/wrong/old source code. In general, building code with a different compiler, or even a different compiler version, can shake out unfortunate bugs like this. I really hate to ruin any good-will between Samsung and the community by making inflammatory acusations when, frankly, they're probably just as puzzled as we are over this.
Best of luck!
mkasick said:
I have a suspicion that what's going on here is actually just a (possibly long-standing) bug. However, the situation is a bit strange all around.
To be specific, I suspect it's a race condition that happens to be triggered by your kernel build, but not Samsung's. This could easily be due to nothing more than using a different compiler, i.e., one that produces more efficient code.
I notice that in "Bad Boot #1" and "Bad Boot #2", the "[MULTIPDP][poll_thread] dpram_open failed!!" message is preceeded by "[MULTIPDP]filp_open() failed~!: -6". Looking at multipdp.c, dpram_open is failing to open "/dev/dpram1" and returning ENXIO ("No such device or address"), indicating that /dev/dpram1 doesn't exist yet.
Now, there's only two references to "/dev/dpram1" in the entire stock kernel image. The first is the line in multipdp.c that it's failing on. The second is in initramfs's /sbin/init. Here, Android's init actually contains a simplified implementation of udev to handle kernel uevents. That is, it's responsible for dynamically creating device nodes as kernel drivers are loaded, one of which is /dev/dpram1.
What I think is happening is this:
init.rc insmods dpram.ko
dpram.ko registers the dpram tty devices, issuing an add uevent message for device 252:2.
... (some time elapses) ...
init processes the uevent message, creates /dev/dpram1.
Meanwhile:
init.rc insmods multipdp.ko
multipdp.ko creates the poll_thread kernel thread.
poll_tread calls dpram_open.
dpram_open attempts to open /dev/dpram1, which might not exist yet.
Probably what's happening is that Samsung's kernel "takes long enough" to get to the "insmod multipdp.ko" part of init.rc that init has time to process the dpram.ko uevents and create /dev/dpram1, so that by the time dpram_open is called, the device node exists. However, your kernel is "fast enough" that init.rc gets to "insmod multipdp.ko", creates the new kernel thread and executes it, before init has caught up on device node creation. Thus, /dev/dpram1 doesn't exist and everything goes to ****.
Now here is where it gets bizarre. If we compare against the Epic's EC05 init.rc, there's an interesting stanza prior to module initialization that's missing here:
Code:
#samsung dpram modules
mknod 0666 /dev/dpram0 c 255 1
mknod 0666 /dev/dpram1 c 255 2
mknod 0666 /dev/dpramerr c 255 0
mknod 0666 /dev/ttyCDMA0 c 240 1
Presumably this is supposed to create the missing device nodes before insmoding both dpram.ko and multipdp.ko, eliminating the race condition. However, mknod is a command not supported by init, there's no mknod program in the initramfs, and those are the wrong device numbers. Perhaps it's an acknowledgment that the race condition exists on both devices, even though nothing is actually done on the Epic to avoid it? It's just, ... strange.
Unfortunately I can't think of a great solution. One option would be to modify multipdp.c to add a sleep loop around "filp_open(DPRAM_DEVNAME, ...)" to wait for the device to be created. However, that would mean using your own compile of multipdp.ko instead of the one Samsung provides, which could open a different can of worms.
A second option is to move "insmod /lib/modules/multipdp.ko" until after mounting /system, hoping that init would've caught up by then. This can even be insured by running a script that checks for the device node and sleeps if it doesn't exist yet. Something like:
Code:
#!/system/bin/sh
for i in 1 2 3 4 5; do
ls /dev/dpram1 && break
sleep 1
done
and call it from init.rc, after mounting /system, with:
Code:
exec /system/etc/wait_for_dpram1.sh
Anyways, even if this particular issue turns out not to be the culprit, I wouldn't be too quick to conclude that Samsung must've released bad/wrong/old source code. In general, building code with a different compiler, or even a different compiler version, can shake out unfortunate bugs like this. I really hate to ruin any good-will between Samsung and the community by making inflammatory acusations when, frankly, they're probably just as puzzled as we are over this.
Best of luck!
Click to expand...
Click to collapse
.....and why are you not a recognized developer? You deserve it more than me.
mkasick, thanks for the reply, we will look into this. EDIT: Keep in mind that we are using the same toolchain as Samsung, specifically CodeSourcery 2009q3.
mkasick said:
I have a suspicion that what's going on here is actually just a (possibly long-standing) bug. However, the situation is a bit strange all around.
To be specific, I suspect it's a race condition that happens to be triggered by your kernel build, but not Samsung's. This could easily be due to nothing more than using a different compiler, i.e., one that produces more efficient code.
I notice that in "Bad Boot #1" and "Bad Boot #2", the "[MULTIPDP][poll_thread] dpram_open failed!!" message is preceeded by "[MULTIPDP]filp_open() failed~!: -6". Looking at multipdp.c, dpram_open is failing to open "/dev/dpram1" and returning ENXIO ("No such device or address"), indicating that /dev/dpram1 doesn't exist yet.
Now, there's only two references to "/dev/dpram1" in the entire stock kernel image. The first is the line in multipdp.c that it's failing on. The second is in initramfs's /sbin/init. Here, Android's init actually contains a simplified implementation of udev to handle kernel uevents. That is, it's responsible for dynamically creating device nodes as kernel drivers are loaded, one of which is /dev/dpram1.
What I think is happening is this:
init.rc insmods dpram.ko
dpram.ko registers the dpram tty devices, issuing an add uevent message for device 252:2.
... (some time elapses) ...
init processes the uevent message, creates /dev/dpram1.
Meanwhile:
init.rc insmods multipdp.ko
multipdp.ko creates the poll_thread kernel thread.
poll_tread calls dpram_open.
dpram_open attempts to open /dev/dpram1, which might not exist yet.
Probably what's happening is that Samsung's kernel "takes long enough" to get to the "insmod multipdp.ko" part of init.rc that init has time to process the dpram.ko uevents and create /dev/dpram1, so that by the time dpram_open is called, the device node exists. However, your kernel is "fast enough" that init.rc gets to "insmod multipdp.ko", creates the new kernel thread and executes it, before init has caught up on device node creation. Thus, /dev/dpram1 doesn't exist and everything goes to ****.
Now here is where it gets bizarre. If we compare against the Epic's EC05 init.rc, there's an interesting stanza prior to module initialization that's missing here:
Code:
#samsung dpram modules
mknod 0666 /dev/dpram0 c 255 1
mknod 0666 /dev/dpram1 c 255 2
mknod 0666 /dev/dpramerr c 255 0
mknod 0666 /dev/ttyCDMA0 c 240 1
Presumably this is supposed to create the missing device nodes before insmoding both dpram.ko and multipdp.ko, eliminating the race condition. However, mknod is a command not supported by init, there's no mknod program in the initramfs, and those are the wrong device numbers. Perhaps it's an acknowledgment that the race condition exists on both devices, even though nothing is actually done on the Epic to avoid it? It's just, ... strange.
Unfortunately I can't think of a great solution. One option would be to modify multipdp.c to add a sleep loop around "filp_open(DPRAM_DEVNAME, ...)" to wait for the device to be created. However, that would mean using your own compile of multipdp.ko instead of the one Samsung provides, which could open a different can of worms.
A second option is to move "insmod /lib/modules/multipdp.ko" until after mounting /system, hoping that init would've caught up by then. This can even be insured by running a script that checks for the device node and sleeps if it doesn't exist yet. Something like:
Code:
#!/system/bin/sh
for i in 1 2 3 4 5; do
ls /dev/dpram1 && break
sleep 1
done
and call it from init.rc, after mounting /system, with:
Code:
exec /system/etc/wait_for_dpram1.sh
Anyways, even if this particular issue turns out not to be the culprit, I wouldn't be too quick to conclude that Samsung must've released bad/wrong/old source code. In general, building code with a different compiler, or even a different compiler version, can shake out unfortunate bugs like this. I really hate to ruin any good-will between Samsung and the community by making inflammatory acusations when, frankly, they're probably just as puzzled as we are over this.
Best of luck!
Click to expand...
Click to collapse
You weed out toolchain, source, and what you have left is a defconfig that probably has more debugging active than the one that's shipped with the source.
No other way to explain size differential.
EDIT: Also, holy awesome on that analysis.
mkasick said:
One option would be to modify multipdp.c to add a sleep loop around "filp_open(DPRAM_DEVNAME, ...)" to wait for the device to be created.
Click to expand...
Click to collapse
Attached is a patch that should implement this. I've also attached a build of the kernel with it applied--given that we might be looking at funny race condition issues, it's probably worth seeing if the build environment has as much of an effect here.
Mind you, I don't have an Indulge and was unable to test. No idea if this actually works, but hopefully it's progress.
Now, if it does work, it's worth running a dmesg and looking for a "/dev/dpram1 does not exist yet, sleeping." just prior to "dpram_open successd !!!!". If that message isn't there, then it wasn't strictly the patch that contributed to a successful kernel, rather it's probably an artifact of the build.
Full disclosure:
The kernel sources are from SCH-R910_OpenSource.zip posted the Samsung open source page. The initramfs was extracted from this Odin repackage of the stock kernel, linked to from androidforums, although in retrospect I probably should've pulled it from k0nane's GitHub. It was compiled with CodeSourcery G++ Lite 2010q1-188 (EABI) on a Debian sid machine last updated sometime two weeks ago.
In additon to the aforementioned patch (indulge_multipdp_wait_for_dpram1.diff), I used another patch (indulge_no_ifdef_linux.diff) to get rid of the single "#ifdef linux" that trips up the eabi compiler and shouldn't be necessary with the gnueabi one. Otherwise, the only other source changes I made were to the CROSS_COMPILE variable in Kernel/Makefile and CONFIG_INITRAMFS_SOURCE in Kernel/arch/arm/configs/forte_01_defconfig to set the appropiate paths for those two.
I build the kernel once with the original initramfs, copied the newly-built modules/samsung/multipdp/multipdp.ko over the existing one in the initramfs, and rebuilt the kernel to link in the updated initramfs. That should be it.
k0nane said:
Keep in mind that we are using the same toolchain as Samsung, specifically CodeSourcery 2009q3.
Click to expand...
Click to collapse
Yes you're right.
I know that for the Epic's DI18 Samsung used what looked to be an in-house build of GCC 4.3.1 for the stock kernel even though they were recommending CodeSourcery in the source README. Apparently they switched to 2009q3-67 for Froyo and I never noticed.
Decad3nce said:
You weed out toolchain, source, and what you have left is a defconfig that probably has more debugging active than the one that's shipped with the source.
No other way to explain size differential.
Click to expand...
Click to collapse
The defconfig on my Github is identical to that of the package on opensource.samsung.com with the exception of LZMA compression being enabled and a path to the initramfs files being set.
mkasick said:
Attached is a patch that should implement this. I've also attached a build of the kernel with it applied--given that we might be looking at funny race condition issues, it's probably worth seeing if the build environment has as much of an effect here.
Click to expand...
Click to collapse
I'll pass this on to an Indulge user, or test it myself if I find a working microUSB cable first. Thanks. EDIT: No such luck! Died within 15s like most kernel builds do. Unfortunate. I can have the user acquire a log, but I am 95% certain no new info will be presented.
k0nane said:
Died within 15s like most kernel builds do.
Click to expand...
Click to collapse
Crud.
k0nane said:
I can have the user acquire a log, but I am 95% certain no new info will be presented.
Click to expand...
Click to collapse
I'd be curious if it contains any more statements about dpram_open, suggesting whether or not that patch had any effect. Or the built multipdp.ko even works.
Either way, if the device is automatically rebooting after 15 seconds, it's either hitting a kernel panic or something funny is going on with the watchdog. If we could get console output with a UART jig that would likely be helpful.
Can (stock) recovery be booted with a source-compiled kernel and survive for some length of time?
mkasick said:
I'd be curious if it contains any more statements about dpram_open, suggesting whether or not that patch had any effect. Or the built multipdp.ko even works.
Click to expand...
Click to collapse
I will acquire a log ASAP. A new copy of that kernel with adbd flipped to root on boot would be useful, as it's difficult to capture a full kmsg without it - I can generate it via your patch, or if you wish to substitute in the initramfs from my github, that's an option as well.
mkasick said:
Either way, if the device is automatically rebooting after 15 seconds, it's either hitting a kernel panic or something funny is going on with the watchdog. If we could get console output with a UART jig that would likely be helpful.
Click to expand...
Click to collapse
And unfortunately, building one is a bit beyond my skill level. I'd be willing to buy one. The jig should work for the Indulge, given it is essentially a Galaxy S device. I agree that it's most likely panicking - it would be helpful if there were some kind of output to the display as there is with Epic kernel panics.
mkasick said:
Can (stock) recovery be booted with a source-compiled kernel and survive for some length of time?
Click to expand...
Click to collapse
That has not yet been tested. There's a /system/bin/recovery and a recovery partition; as with the Epic and other devices, the partition should be triggered by the button combination on boot. I'll have a user flash a built kernel to bml9.
k0nane said:
or if you wish to substitute in the initramfs from my github, that's an option as well.
Click to expand...
Click to collapse
Attached is a build with your github initramfs.
k0nane said:
And unfortunately, building one is a bit beyond my skill level. I'd be willing to buy one. The jig should work for the Indulge, given it is essentially a Galaxy S device.
Click to expand...
Click to collapse
I have to read through that thread again, but I believe the simple version of the JIG is a 3.3v USB-to-serial converter along with a single resistor to activate the UART. I have some FT232 converters lying around, but they're 5v.
I'll order some parts and see if I can build one for myself in a week or so, as it would be useful for Epic debugging. If it works, we can work something out to get one sent your way.
k0nane said:
That has not yet been tested. There's a /system/bin/recovery and a recovery partition; as with the Epic and other devices, the partition should be triggered by the button combination on boot.
Click to expand...
Click to collapse
I'm thinking ~15 seconds is a somewhat strange time to die during boot. If recovery (with adb) runs stable, then the kernel works in a minimal environment. Perhaps some narrowing down of the boot sequence can be done from there?
Sorry to derail this but I'm cross posting here..
I sent an inquiry to the EFF to see if this is an issue that they can help with. Below is the response I received:
Rebecca said:
Rebecca S. Reagan [email protected] to me
show details 6:27 PM (12 hours ago)
Hi there!
Thank you for contacting the Electronic Frontier Foundation (EFF).
Unfortunately, we do not have the capacity necessary to take on open source issues like this one. The Electronic Frontier Foundation (EFF) is a small, grassroots legal advocacy non-profit focused on defending civil liberties and constitutional rights online with very limited resources. As such, we have a very important but narrowly focused mission and are not equipped to address all of the potential issues that escape our limited scope.
Regards,
Rebecca
Click to expand...
Click to collapse
I took a second look, found, and fixed another problem. Attached are more builds.
I compiled an unpatched kernel using 2009q3-67, decompressed it, and ran "strings" against both it and the stock kernel. I diffed the strings output to look for anything missing that I could grep for in the source tree. I found some things in "Kernel/arch/arm/mach-s5pv210/mach-aries.c" in the stock kernel that were missing from my build.
First were missing definitions for "dpram-device", "onedram", and "modemctl". The second were strings corresponding to the modemctl_cfg_gpio function, which is responsible for setting the "PHONE_ON", "CP_RST", and "PDA_ACTIVE" gpios. For these items to be included requires that CONFIG_SAMSUNG_PHONE_TTY and CONFIG_SAMSUNG_PHONE_SVNET be set in the kernel config file. Which they are (as "=m") in forte_01_defconfig.
Here's the problem: when build_kernel.sh runs "make defconfig", the both config options are missing in the resulting .config file. It turns out that as part of the reorganization that Samsung did to package the source release, they moved the onedram drivers from the Kernel source directory to the external module one, and dropped the Kconfig files that define these options. Without them, the build infrastructure assumes that all (useful) references those config options are removed from the all source files and purges them from the configuration. Except they're still needed by mach-aries.c, and without them, a key part to the kernel radio interface goes missing.
This definitely contributes to why the radio can't be brought up on boot. This may also cause the reboot after 15 seconds, although I'm not certain.
Anyways, I fixed this problem by adding stub entires for CONFIG_SAMSUNG_PHONE_TTY and CONFIG_SAMSUNG_PHONE_SVNET to Kernel/arch/arm/Kconfig, which allows those options to be preserved by "make defconfig". I've also verified that all the other purged config options aren't used in this source tree.
Attached is a patch containing the modified Kconfig, just apply and clean build as normal. I've also attached two kernel builds for testing, one with the stock initramfs and the other with k0nane's.
Some more notes:
While comparing strings, I noticed that Samsung appears to do some amount of source clean-up and reorganization in preparation of open source packages. For example, there's a bunch of files stored in a "forte" subdirectory for whatever driver that are moved up to the parent, e.g., "Kernel/arch/arm/mach-s5pv210/forte/mach-aries.c" is moved to "Kernel/arch/arm/mach-s5pv210/mach-aries.c". This alone shouldn't be a problem, but if it was done improperly, i.e., not all forte-specific changes were pushed to the parent, some functionality or device-specific bug fixes could be lost.
With regard to my earlier concern about there being a race condition, I'm uncertain at this point if it's actually a problem. It's possible that the missing device definitions are picked up from sysfs by init, and the relevant device nodes created that way. I haven't verified that this is the case, but given that the Epic should be vulnerable to the same condition, but we never observe it, perhaps it's not really an issue. Also, the reason why modules compiled from the source package are much larger than the ones shipped with the initramfs is that they're unstripped. Compiling with the "INSTALL_MOD_STRIP" environment variable defined results in them being stripped upon "make modules_install".
Hope that helps.
mkasick, you are a god. You've done the work that Samsung completely neglected to do - whether deliberately or not.
I suppose this thread can be closed, now. The changes have been pushed to my Github.
Thanks everyone for your input, and mkasick for fixing an obscure problem on a device you don't own!
Team Cody!
Member List (in no particular order):
@thewisenerd
@omerjerk
@#Superuser
@dhrumangajjar
@navinn
@xaak
@Agaphetos
WANTED
- Testers - to beat the hell out of our ROMs
- Graphic Designers - boot logos, wallpapers, banners(one or two people)
- Themers - theme EVERYTHING!!!
- Co-Dev(s) - Do i really know what I'm doing most of the time (the answer to that is "probably not")
- Moral Support - LOL
- Suggestions - anything you got - Lay It On Meh!!
our works:
MiniCM9
CodyROM
CM9 Build #15
Evervolv
AOKP
OmniROM
AOSPA-legacy
note to self: add all the works, links
find a thread tagged [TeamCody], that's ours
Team-
@thewisenerd
@omerjerk
@#Superuser
@dhrumangajjar
@navinn
Source- https://www.github.com/CodyROM/
Blog- http://www.codyrom.wordpress.com/
Yea, we need support and suggestions!
todo list:
generalise this
First, finish the work on "reverse mounter"; as it is *technically speaking* easy (for me, to write scripts, that do things); and is pretty straight forward (no aimless edits or so)
fix gps in omni
fix video recording in omni
udpate twrp
switch to wlan0 in omni
random trolling lol
@thewisenerd
we should fix the date bug in notif bar..
is that xml derps..??
navinn said:
@thewisenerd
we should fix the date bug in notif bar..
is that xml derps..??
Click to expand...
Click to collapse
No, its a java derp
Compiling CM11/Kitkat for pico!
I assume you have knowledge of building with source, even if not, I don't really care (for there are a lot of guides).
First, get *all* the required packages installed. As for what I mean by *all*, is not really a question for me to answer. I used <insert-random-guide-from-xda> here, and the source.android.com's guide on initializing, just to be sure.
As for java, I use openjdk. *most* sources today support openjdk, elseways, either, you could force it to be compatible by removing just ONE line in build/core/main.mk
So, that's the packages part.
As for syncing sources, as most guides say, you DON'T need to use a ~/system folder or anything. You can use a folder, anywhere.
Just open up a shell, wherever you want to get the sources synced, and type in the following commands.
I assume you have the repo tool installed. IMHO, you should really create a "bin" folder in your homefolder (short linked as "~/") using the following commands:
Code:
mkdir -p ~/bin
curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
Then, open up your ~/.bashrc file, using Geany (a better alternative to Gedit, perhaps ), using the command:
Code:
geany ~/.bashrc
Just insert this line, randomly (preferrably at the end of the file).
Code:
PATH=$PATH:/home/<insert-your-username>/bin
Now, for syncing sources. Make sure you are a directory above working folder.
Code:
mkdir -p <insert-ROM-name>
cd <insert-ROM-name>
Now, that you are inside the working folder, take your pick about the ROM that you are going to compile. This tutorial is for CM11, so i'd be running the following command.
Code:
repo init -u git://github.com/CyanogenMod/android.git -b cm-11.0
Yes, you can reference this from a cm10.2 source that you've already downloaded, and this will reduce the size required to download. Why, you could even reference this from an ICS source (just saying, not meant to be even tried).
If you aren't really building for mac, I'd suggest that you don't download the darwin repos.
Short way that I use:
Run following command in working folder.
Code:
mkdir -p .repo/local_manifests
grep "darwin" .repo/manifests/default.xml > .repo/local_manifests/local_manifest.xml
Open up local_manifest.xml using
Code:
geany .repo/local_manifests/local_manifest.xml
Add these lines on top:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
and the following line at end.
Code:
</manifest>
Now open up "search and replace" dialog box.
Search for:
Code:
<project
and replace with:
Code:
<remove-project
Voila, you've removed all the unnecessary darwin repos.
Now, for repo syncing.
I'd seriously suggest that if you live in parts of the world where the internet speeds are less than 1 m'b'ps (~128 k'B'ps (worst case, mine's around 60 k'B'ps, and I've known people with speeds ranging 30 k'B'ps), you'd sync just one repo at a time (the -jx).
run:
Code:
repo sync -j1
Else, if you have a super fast internet connection, please don't brag. Use a higher number, preferably 3, or 4.
Now, sync these repos, while I attend my classes. I'd return back and write part 2.
edit: part 2; go here: http://forum.xda-developers.com/showpost.php?p=49349691&postcount=8
Compiling CM11/Kitkat for pico! PART 2
reserved
Also, yes, you could use this localmanifest here: https://github.com/PicoKat/local_manifest/blob/dev/local_manifest.xml
but, you'd miss out the learning process
Anyways, this post would take up part 2 of the tutorial.
This is purely device specific. and, there's already a guide on android development section (of this device), the reason why I didn't make a new thread.
part 2 begins here:
Ok, so, you'd synced the repos?
Now, as opposed to roomservice, I'd prefer manually cloning the device tree, kernel, and vendors.
Clone a device tree (galaxyfreak's, or mine), into /device/htc/pico.
Clone the kernel sources from here: https://github.com/PicoKat/android_kernel_htc_pico
Clone the vendor tree from here: https://github.com/PicoKat/android_vendor_htc_pico
(note: if you are using my device tree, that has ION, you'd have to use my vendor tree, with updated blobs here: https://github.com/vineethraj49/android_vendor_htc_pico)
Ok, so that part's setup.
Now, lets move on to cherry-picking, and/or patching some stuff (from legaCyMod).
Note: you could have just added these to local_manifest too...
android_build:
Hopefully, you should know how to add remotes, and working with them, if not, click on the show content, below (as a sample for android_build):
Code:
git remote add legacymod git://github.com/legaCyMod/android_build.git
git fetch legacymod cm-11.0
git cherry-pick <commit-id>
where commit id's the looong "number" following /commit/ in the URL.
1. always rebuild build.prop
2. bringing back squisher
3. Removing TTS data
4. Revert adding auditd
5. add kitkat sdk versions
6. Revert "add drawables for all densities" (seriously, this saves space.)
7. adding support for our recovery
frameworks/av:
1. enable meta mode for video msm7x27a
2.bring back support for legacy omx
3. support legacy qcom audio variant
vendor/cm:
1. bring back squisher
2. revert adb authentication (yea, screw adb while dev'ving
3. remove ze bloats
Next, clone media-legacy to hardware/qcom/media-legacy, and display-legacy to hardware/qcom/display-legacy.
waait!!!!
You need to do some patches in some files. required to fix up camera.
patch two files, manually, making some sense out of this:
frameworks/base: https://github.com/szezso/vivo_cm11_patches/blob/d819363fe7d181e73841d3fa35c1b8c0c7d7c046/frameworks_base.patch
frameworks/av: https://github.com/szezso/vivo_cm11_patches/blob/2f1c87f4453c1c7c88e3c7d2b011994f894fe669/frameworks_av.patch
The next step:
Get prebuilts, seriously.
Code:
cd vendor/cm
./get-prebuilts
Then, "cd" back to root of source dir.
Run the following commands:
Code:
. build/envsetup.sh
brunch pico -j4
Hopefully, you should see an output package by the end of <insert-time-directly-propotional-to-crappiness-of-computer>
For batch-resizing bootanimation (in linux):
resizing part:
Code:
for file in *.png; do convert $file -resize 320x320! $file; done
you'd know which part to edit
for zip'ing part:
Code:
zip -r -y -q -0 bootanimation.zip *
will edit this later
Using CCahe
Now something very important who gonna compile ROMs, use CCache. It'll spped up the building process.
CCache - Compiler cache
How to use?
> Open a Terminal(Ctrl+Alt+T)
>
Code:
gedit /.bashrc
[You can use any alternative to gedit )
> Add the following lines to it:
Code:
export USE_CCACHE=1
export CCACHE_DIR=/[COLOR="Red"]source[/COLOR]/prebuilts/misc/linux-x86/ccache
Source here is the folder in which you have the code downloaded.
>
Code:
cd source
>
Code:
prebuilts/misc/linux-x86/ccache/ccache -M [COLOR="red"]25G[/COLOR]
25G = 25 GB of space to be used as CCache. You can use any value in this.
How does ccache work ?
Well , in short , what it does is that for C and C++ programs it caches the output. One it detects that the program is getting repeated it directly sends out the output and thus reduces the time of compilation.
After using ccache you will really feel the difference in the compilation speeds of your ROMs and kernels. //Copied.
Credits - Red Devil.
I use CCACHE, but the only difference is that I don't specify the directory. also, I did a "apt-get install ccache" long ago, and set file size according to that. Its still working just as fine.
and oh, yes, adding that USE_CCACHE=1 to .bashrc is required
IRC and you!
In case you came here via my signature, sorry for that. But really, you need to know some stuff about IRC, before going to a channel. The pico channel is #pico :fingers-crossed:
So, you might have heard about IRC, even logged into one (freenode, probably, the webclient of it), (and most probably saw that there was no activity and just closed that browser tab).
Let's take a timeout here to talk about IRC.
Instead of me talking about IRC, (and you being too lazy to Google stuff (not intended to those who solve problems themselves (not noobs (?) :wink: :wink)), here's a nice article by Rey Bango that you might want to read: IRC is Back: Here's Your Starter Guide
So, instead of me talking blah blah, and you skimming through most parts, which I assume, you did in the above article (considering you taking the energy to read it), here's what you need to know about Freenode:
..but in terms of development, from my experience, most developers tend to jump on Freenode - and rightfully so.
Click to expand...
Click to collapse
So, just to let you know, in those channels (where you thought there was no activity), there actually is a lot of discussion going on. You just don't know it yet.
So? What am I supposed to do? Keep a browser tab open? Don't expect me to say "Exactly not!" because that too is a viable option, though not very practically possible.
Here's where bouncers come in. Known as BNCs, here's what they do:
A bouncer (BNC for short) is a piece of software used to relay the communication between an IRC client and the network it is connected to, acting as a Proxy.
So, whats the point you ask? The reasons and benefits of using a bouncer are many and include hiding the real IP you are connecting from, protecting your nickname and channel from being taken on networks that don’t provide channel-/nickname registration and most of the time they’ll also notify you of private messages that came in when you’ve been disconnected – of course only when the bouncer can stay online being connected from a server.
Click to expand...
Click to collapse
Coming as either free or paid (either of them being useful, nonetheless), and cutting all the tech jargon, they allow you to "stay online" allowing you to close that browser tab you feared to keep open.
So, in case you actually read through the above quoted text, software!? Being connected from a server? Nah, just kidding. Here's where online BNC providers come in. They provide you a username, (and of course a password, which you need to select), and they host for you the BNC software (in most cases, ZNC).
Most BNCs allow caching of messages, i.e. buffered playback. Their servers are connected 24x7. You, the user, connect to their server, and through their server join the channels. You then, disconnect from their channel. Poof! But, their server still remains connected to the channels, and logs the messages in the channels, and any PMs sent to you. The next time you connect to their server, it gives you all the logged stuff, and it gets cleared. The next time you disconnect, ... Well, you get the flow.
Questions. That's what you go to IRCs for (mostly. I do it for the conversations ). How exactly do you ask questions? Similar to on XDA, but slightly different. I couldn't have said better than l3dx's answer on stackexchange:
Rule #1: Don't ask to ask
Rule #2: Behave as you would do in a real life conversation
Rule #3: Be patient. If there is no activity, it usually means that no one has read what you wrote yet. If no one responds, they don't know or didn't notice. You can re-try after a while, or ask if anyone has any clue with regards to your question x minutes ago.
Click to expand...
Click to collapse
What does Rule #1 mean?
Something very similar to what zxcdw said in the comments:
"Don't ask to ask. Just ask". Don't ask people "Anybody around?" or let alone highlight others to ask if they are around. It's just easier to drop your question, hang around and wait. Ask again in a few hours or so.
Click to expand...
Click to collapse
Also, use English, in most channels, if possible. Stay safe, don't open any unknown links or links to untested software, scripts and stuff. Stay safe.
IRC clients help you stay connected. A few simple google searches will get you going in the right direction. I personally use XChat IRC on Linux, but there are a lot of other good IRC clients available too.
Free bouncers should do, mostly. I use bnc.im, seen people using EliteBNC, some people installing ZNC in their own private servers that are online 24x7, YMMV.
That's about it for IRCs. Happy chatting!
P.S. Make sure you ask the right questions in the right channels.
sys2cache... something to help with less /system partition size.
note: this would be running at boot, with busybox, so,...
Code:
# algo:
# if read(switch_file=1)
# goto case 2.
# else case 1:
# get /system size
# get /data size
# get /cache size
# check /cache remaining size
# check for files in /system
# check for the following folders
# name generic size priority
# ./etc 3.1M 6
# ./vendor 36K 9
# ./addon.d 12K 8
# ./core 76K 7
# ./xbin 1.6M 5
# ./fonts 18M 1
# ./usr 21M 2
# ./media 4.8M 3
# ./bin 5.5M 4
#
# switch = 0;
# file folder_list;
# for i in list_args
# if (getsize($i) < getfreesize(/cache))
# mv $i /cache/$i
# mount bind /cache/$i /system/$i #fix syntax
# ifexists(/system/$i)
# switch = 1;
# folder_list >> $i
# else
# switch = 0; break;
# else
# echo "fak this, we're out of space!"
# fi
#
# echo $switch > switch_file
#
# sync; exit!
#
#
#
# case 2:
#
# for i in xargs(folder_list)
# do
# mount bind /cache/$i /system/$i #fix syntax
# done
#
# sync; exit!
#
#
#
# todo:
# * actually code this shiz
# * better error checking
# * add logging
Hey!
Make sure y'all check this!
http://team-cody.github.io/ :highfive:
welcome to another member @Agaphetos
Source- https://www.github.com/Team-Cody/
todo: update this post with website link.
lol, nvm this post
shameless bump.
since my board exams are over (sort of; only CS is left); i thought of building something for pico (a looong time since I did so).
Waht do I build?
At first thought; one *pure* CM9 build with *some* cherry-picks;
another; an updated CM11 build (updated sources and stuff)...
fixing up omx and stuff on Omni is on the "to-do" list and going to take some 'time' to do; so don't expect it to be fixed already :silly:
So; which one do I 'invest' time in? :cyclops:
@thewisenerd
MTP and OTG in cm9
Cm11 only if you can make it compressed
Sent from my Xperia S using XDA Free mobile app
mirhl said:
@thewisenerd
MTP and OTG in cm9
Cm11 only if you can make it compressed
Sent from my Xperia S using XDA Free mobile app
Click to expand...
Click to collapse
you mean MTP && USB tethering?
edit: a test build will be up today (if build goes well, that is).
edit: here you go: cm-9-20150315-UNOFFICIAL-pico.zip
thewisenerd said:
you mean MTP && USB tethering?
edit: a test build will be up today (if build goes well, that is).
edit: here you go: cm-9-20150315-UNOFFICIAL-pico.zip
Click to expand...
Click to collapse
This is one sick ROM - good job !!!
Hello dear developers.
I'd like to share with you effect of nearly 300 hours spent on trying to optimize Android and push it to the limits.
In general. You should be already experienced in setting up your buildbox, using git, building AOSP/CyanogenMod/OmniROM from source and cherry-picking things from review/gerrit. Solving git conflicts would also be nice. If you don't know how to build your own ROM from source, this is not a something you can apply to your ROM. Also, as you probably noticed, this is not a something you can apply to already prebuilt ROM (stocks), as these optimizations are applied during compilation, so only AOSP roms, self-compiled from source may use this masterpiece.
So, what is it about? As we know, Android contains a bunch of low-level C/C++ code, which is compiled and acts as a backend for our java's frontend and android apps. Unfortunately, Google didn't put their best at focusing on optimization, so as a result we're using the same old flags set back in 2006 for Android Donut or anything which existed back then. As you guess, in 2006 we didn't have as powerful devices as now, we had to sacrifice performance for smaller code size, to fit to our little devices and run well on very low amount of memory. However, this is no longer a case, and by using newest compilers and properly setting flags, we can achieve something great.
You probably may heard of some developers claiming using of "O3 Flags" in their ROMs. Well, while this may be true, they've applied only to low-level ARM code, mostly used during kernel compilation. Additionally it overwrites O2 flag, which is already fast, so as you may guess, this is more likely a placebo effect and disappears right after you change the kernel. Take a look at the most cherry-picked "O3 Flags commit". You see big "-Os" in "TARGET_thumb_CFLAGS"? This is what I'm talking about.
However, the commit I'm about to present you is not a placebo effect, as it applies flags to everything what is compiled, and mostly important - target THUMB, about 90% of an Android.
Now I'll tell you some facts. We have three interesting optimization levels. Os, O2, O3. O2 enables all optimizations that do not involve a space-speed tradeoff. Os is similar to O2, but it disables all flags that increase code size. It also performs further optimizations to reduce code size to the minimum. O3 enables all O2 optimizations and some extra optimizations that like to increase code size and may, or may not, increase performance. If you want to ask if there's something more like O4, there is - Ofast, however it breaks IEEE standard and doesn't work with Android, as i.e. sqlite3 is not compatible with Ofast's -ffast-math flag. So no go for us.
Now here comes the fun part. Android by default is compiled with O2 flag for target ARM (about 10% of Android, mostly low-level parts) and Os flag for target THUMB (about 90% of Android, nearly everything apart from low-level parfts). Some guys think that Os is better than O2 or O3 because smaller code size is faster due to better fitting in cpu cache. Unfortunately, I proven that it is a myth. Os was good back in 2006, as only with this flag Google was able to compile Dalvik and it's virtual machine while keeping good amount of free memory and space on eMMC cards. As or now, we have plenty of space, plenty of ram, plenty of CPU power and still good old Os flag for 90% of Android.
I've made countless tests to find out what is the most efficient in terms of GCC optimization, two selected tests I am about to present you right now.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
As you may noticed, I compiled whetstone.c benchmark using three different optimization flags - Os, O2 and O3. I set CPU to performance, maximum frequency, and I repeated each test additional two times, just to make sure that Android doesn't lie to me. Source code of this test is available here and you may download it, compile for our beloved Android and try yourself. As you can see O3 > O2 >> Os, Os performs about 2.5x times worse than O2, and about 3.0x times worse than O3.
But, of course. Android is not a freaking benchmark, it's operating system. We can't tell if things are getting better or worse according to a simple benchmark. I kept that in mind and provided community with JustArchi's Mysterious Builds for test. I gave both mysterious builds and didn't tell my users what is the mysterious change. Both builds have been compiled with the same toolchain, same version, same commits. The one and only mysterious change was the fact that every component compiled as target thumb (major portion of an android) has been optimized for speed (O3) in build #1 (experimental), and optimized for size (Os) in build #2 (normal behaviour). Check poll yourself, 9 votes on build 1 in terms of performance, and 1 vote on build 2. I decided that this and benchmark is enough to tell that O2/O3 for target thumb is something that we want.
Now it doesn't matter that match if you wish to use O2 or O3, but here is some comparison:
1. Kernel compiled with O2 has 4902 KB, with O3 4944 KB, so O3 is 42 KB bigger.
2. ROM compiled with O3 is 3 MB larger than O2 after zip compression. Fast overview: 97 binaries in /system/bin and 2 binaries in /system/xbin + 283 libraries in /system/lib and other files, about 400 files in total. 3 MB / 400 = 7,5 KB per file size increase.
3. It's unlikely that code working properly with O2 level might break on O3 level, most issues are on the Os <-> O2 part.
4. If it doesn't cause any issues, and speeds up a binary by a little bit, why not use it?
5. The only real reason to not use O3 is potential higher memory usage due to oversized binaries.
In general, I doubt that this extra chunk of code may cause any significant memory usage or slower performance. I suggest to use O3 if it doesn't cause any issues to you compared to O2, but older devices may use O2 purely for saving on code size, similar way Google did it back in 2006 using Os flag.
[SIZE="+1"]Now let's get down to business[/SIZE].
Here is a list of important improvements:
- Optimized for speed yet more all instructions - ARM and THUMB (-O3)
- Optimized for speed also parts which are compiled with Clang (-O3)
- Turned off all debugging code (lack of -g)
- Eliminated redundant loads that come after stores to the same memory location, both partial and full redundancies (-fgcse-las)
- Ran a store motion pass after global common subexpression elimination. This pass attempts to move stores out of loops (-fgcse-sm)
- Enabled the identity transformation for graphite. For every SCoP we generate the polyhedral representation and transform it back to gimple. We can then check the costs or benefits of the GIMPLE -> GRAPHITE -> GIMPLE transformation. Some minimal optimizations are also performed by the code generator ISL, like index splitting and dead code elimination in loops (-fgraphite -fgraphite-identity)
- Performed interprocedural pointer analysis and interprocedural modification and reference analysis (-fipa-pta)
- Performed induction variable optimizations (strength reduction, induction variable merging and induction variable elimination) on trees (-fivopts)
- Didn't keep the frame pointer in a register for functions that don't need one. This avoids the instructions to save, set up and restore frame pointers; it also makes an extra register available in many functions (-fomit-frame-pointer)
- Attempted to avoid false dependencies in scheduled code by making use of registers left over after register allocation. This optimization most benefits processors with lots of registers (-frename-registers)
- Tried to reduce the number of symbolic address calculations by using shared “anchor” symbols to address nearby objects. This transformation can help to reduce the number of GOT entries and GOT accesses on some targets (-fsection-anchors)
- Performed tail duplication to enlarge superblock size. This transformation simplifies the control flow of the function allowing other optimizations to do a better job (-ftracer)
- Performed loop invariant motion on trees. It also moved operands of conditions that are invariant out of the loop, so that we can use just trivial invariantness analysis in loop unswitching. The pass also includes store motion (-ftree-loop-im)
- Created a canonical counter for number of iterations in loops for which determining number of iterations requires complicated analysis. Later optimizations then may determine the number easily (-ftree-loop-ivcanon)
- Assumed that loop indices do not overflow, and that loops with nontrivial exit condition are not infinite. This enables a wider range of loop optimizations even if the loop optimizer itself cannot prove that these assumptions are valid (-funsafe-loop-optimizations)
- Moved branches with loop invariant conditions out of the loop (-funswitch-loops)
- Constructed webs as commonly used for register allocation purposes and assigned each web individual pseudo register. This allows the register allocation pass to operate on pseudos directly, but also strengthens several other optimization passes, such as CSE, loop optimizer and trivial dead code remover (-fweb)
- Sorted the common symbols by alignment in descending order. This is to prevent gaps between symbols due to alignment constraints (-Wl,--sort-common)
Click to expand...
Click to collapse
Sound cool, doesn't it? Head over to my ArchiDroid project and see yourself how people react after switching to my ROM. Take a look at just one small example, or another one . No bullsh*t guys, this is not a placebo.
However, please read my commit carefully before you decide to cherry-pick it. You must understand that Google's flags weren't touched since 7 years and nobody can assure you that they will work properly for your ROM and your device. You may experiment with them a bit to find out if they're not causing conflicts or other issues.
I can assure you that my ArchiDroid based on CM compiles fine with suggested steps written in the commit itself. Just don't forget to clean ccache (rm -rf /home/youruser/.ccache or rm -rf /root/.ccache) and make clean/clobber.
You can use, modify and share my commit anyway you want, just please keep proper credits in changelogs and in the repo itself. If you feel generous, you may also buy me a coke for massive amount of hours put into those experiments.
Now go ahead and show your users how things should be done .
Cherry-picking time!
Android "Lollipop" (5.1.1 & 5.0.2 tested)
JustArchi's ArchiDroid Optimizations V4.1 for CyanogenMod (latest)
A set of commits you may want to pick to fix O3-related issues:
external_bluetooth_bluedroid | hardware_qcom_display | libcore | frameworks_av #1 | frameworks_av #2
Older entries are provided for reference only. I suggest using only latest commit above.
Android "Lollipop" (5.1.1 & 5.0.2 tested)
JustArchi's ArchiDroid Optimizations V4 for CyanogenMod
Android "Kitkat" 4.4.4:
JustArchi's ArchiDroid Optimizations V3 for CyanogenMod
JustArchi's ArchiDroid Optimizations V3 for OmniROM
JustArchi's ArchiDroid Optimizations V2
JustArchi's ArchiDroid Optimizations V1
AFTER applying above commit and AFTER EVERY CHANGE regarding flags, ALWAYS make clean/clobber AND empty ccache (rm -rf ~/.ccache)
Q: How to properly change toolchains used in local manifest?
Open from your source rootdir .repo/local_manifests/roomservice.xml (or create one). Here is a sample manifest that replaces default 4.8 toolchain (both eabi and androideabi) with 4.8 SaberMod and 4.9 ArchiToolchain:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remove-project name="platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8" />
<project name="ArchiDroid/Toolchain" path="prebuilts/gcc/linux-x86/arm/arm-eabi-4.8" remote="github" revision="architoolchain-5.2-arm-linux-gnueabihf" />
<remove-project name="platform/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8" />
<project name="ArchiDroid/Toolchain" path="prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8" remote="github" revision="uber-4.9-arm-linux-androideabi" />
</manifest>
This is only an example, you should use the toolchains that suit you best. My ArchiDroid/Toolchain github repo is a good start to test various different toolchains and decide which one you like the most, or which one causes the least problems for you. I do not suggest any other magic tricks to include custom toolchains, putting your selected one in proper path is enough, avoid magic android_build modifications.
[size=+1]Troubleshooting[/size]
Q: Compiler errror:
Code:
(...)/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8/bin/../libexec/gcc/arm-linux-androideabi/4.8.x-sabermod/cc1: error while loading shared libraries: libcloog-isl.so.4: cannot open shared object file: No such file or directory
This error can be fixed by installing missing library. libcloog-isl.so.4 is provided by libcloog-isl4 package, so on debian-like OSes, you should be able to fix it with:
Code:
apt-get install libcloog-isl4
Q: Compiler errror:
Code:
(...)/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8/bin/../libexec/gcc/arm-linux-androideabi/4.8.x-sabermod/cc1: error while loading shared libraries: libisl.so.13: cannot open shared object file: No such file or directory
This error is very similar to above, but considers other shared library. libisl.so.13 is provided by libisl13 package. Now the problem is that this package is in testing/sid, so we'll need to install it from there.
Add to your /etc/apt/sources.list following entries:
Code:
deb http://ftp.debian.org/debian testing main contrib non-free
deb-src http://ftp.debian.org/debian testing main contrib non-free
Then apt-get update && apt-get install libisl13.
Issues below are for older commits and should be used for reference only
Kitkat THUMB O2+ errors?
These are the most common issues.
* Change -O3 flag from TARGET_thumb_CFLAGS back to -Os, make clean/clobber, empty ccache and try again. This fixes most of the issues.
* RIL problems for for the Exynos 4210 family? Add -fno-tree-vectorize to TARGET_thumb_CFLAGS.
* Broken exFAT -> https://github.com/JustArchi/android_external_fuse/commit/78ebbc4404de260862dca5f0454bffccee650e0d
Errors caused by toolchain?
1. Try Google's GCC 4.8 if you used Linaro 4.8 or SaberMod 4.8
2. Fallback to Google's GCC 4.7 if above didn't help (change TARGET_GCC_VERSION back to 4.7)
Errors caused by GCC 4.8+?
* ART Fix (bootloop) -> https://github.com/JustArchi/android_art/commit/71a0ca3057cc3865bd8e41dcb94443998d028407
* Not booting kernel -> https://github.com/JustArchi/androi...mmit/9262707f4ea471acf40baa43ffe4bfb3cff64de9 and https://github.com/JustArchi/androi...mmit/41a70abcdad746d9415f3ee40f90528feb0c9bdd
Errors caused by GCC 4.9+?
* Graphical glitches in PlayStore -> https://github.com/JustArchi/android_external_webp/commit/36c6201fbb108d6b757f860e2cd57f3982191662
Errors caused by Linaro?
* error: unknown CPU architecture -> https://github.com/JustArchi/androi...mmit/5e5e158a7c147725beae1eeb6785174baacecb03 (Keep in mind that this is a sample fix for smdk4412 kernel, you may need to use similar solution in your own case. Also, this error happens only with Linaro toolchain, doesn't happen with Google's GCC)
Other errors?
* error: undefined reference to 'memmove' -> https://github.com/XperiaSTE/androi...mmit/679a4e571ef77f47892a785e852d8219c1e6807a
[size=+1]Credits[/size]
@IAmTheOneTheyCallNeo - For inspiration and first steps
@metalspring - For some nice commits
@sparksco - For SaberMod, some nice commits and support for the optimization idea
Bravo! I see flags in here that I am yet to try. Thanks for all your work and dedication to this effort.
Going to do a comparison this week against my "neofy" initiative
Thanks!
-Neo
Forum Moderator
This is amazing man!
Enviado desde mi Moto G mediante Tapatalk
Damn... Wish I had an international gs3. I'd be on SlimKat, find your thread and be like:
I'm gonna be 20 years old in about an hour... Should I ask?
Great work :highfive:
I saw you had already a custom 4.10 linaro. If I use it, do I need to change anything on your commit?
Thanks
Sent from my SAMSUNG-SGH-I747 using Tapatalk
_MarcoMarinho_ said:
Great work :highfive:
I saw you had already a custom 4.10 linaro. If I use it, do I need to change anything on your commit?
Click to expand...
Click to collapse
It won't boot, stl port has segfaults when using GCC 4.9+. If you want to use it, sure, change TARGET_GCC_VERSION and include proper toolchain.
JustArchi said:
It won't boot, stl port has segfaults when using GCC 4.9+. If you want to use it, sure, change TARGET_GCC_VERSION and include proper toolchain.
Click to expand...
Click to collapse
Is this the best toolchain [stability/performance] to compile a ROM?
https://github.com/JustArchi/Linaro/tree/4.8-androideabi
_MarcoMarinho_ said:
Is this the best toolchain [stability/performance] to compile a ROM?
https://github.com/JustArchi/Linaro/tree/4.8-androideabi
Click to expand...
Click to collapse
Yes.
JustArchi said:
Yes.
Click to expand...
Click to collapse
Just one last question. Do I need to configure envsetup.sh to the custom toolchain directory?
_MarcoMarinho_ said:
Just one last question. Do I need to configure envsetup.sh to the custom toolchain directory?
Click to expand...
Click to collapse
No, but you need to add Linaro entries to your local_manifest.
https://github.com/JustArchi/ArchiDroid/blob/2.X-EXPERIMENTAL/__dont_include/roomservice.xml#L3
Thank you @JustArchi
i make a build for Slimkat (nexus 5)
Everything is fine but just a noobish question,
I have to enable "TARGET_USE_O3=true" in BoardConfig.mk ?
micr0g said:
Thank you @JustArchi
i make a build for Slimkat (nexus 5)
Everything is fine but just a noobish question,
I have to enable "TARGET_USE_O3=true" in BoardConfig.mk ?
Click to expand...
Click to collapse
No, you shouldn't have any commits apart from mine. It already specifies O3 for everything.
Can I use a 4.7 toolchain with the ArchiDroid optimizations? I've tried compiling my kernel with different 4.8 toolchains before, and it's always resulted in boot loops.
Codename13 said:
Can I use a 4.7 toolchain with the ArchiDroid optimizations? I've tried compiling my kernel with different 4.8 toolchains before, and it's always resulted in boot loops.
Click to expand...
Click to collapse
Actually you can. Just change TARGET_GCC_VERSION back to 4.7.
Well done mate, I'll try contributing from my side to this project
great work archi !! also after porting a fully working 4.4.2 touchwiz to i9300 is it possible to make aosp for i9300 more stable now?
@JustArchi which flags could be used for kernel compiling? And where should I put it in in the Makefile? I don't compile the complete ROM because I have to low machine for that, but I am developing a custom kernel for my Xperia Z. BTW Fajnie trafić na innych Polaków
dragonnn said:
[B @JustArchi[/B] which flags could be used for kernel compiling? And where should I put it in in the Makefile? I don't compile the complete ROM because I have to low machine for that, but I am developing a custom kernel for my Xperia Z. BTW Fajnie trafić na innych Polaków
Click to expand...
Click to collapse
Ktoś tu mówił o Polakach?
Świetna robota @JustArchi :highfive:
dragonnn said:
@JustArchi which flags could be used for kernel compiling? And where should I put it in in the Makefile? I don't compile the complete ROM because I have to low machine for that, but I am developing a custom kernel for my Xperia Z. BTW Fajnie trafić na innych Polaków
Click to expand...
Click to collapse
If you have whole ROM tree then you cherry-pick this commit, lunch your target and make bootimage. This is enough.
If you have standalone kernel, take a look at main Makefile .
Hi,
1. Android source code has multiple directories with code for different devices. Also, within a particular directory, there are paths showing different branches and tags. Just as an example, within 'android / platform / external / iptables' directory (url: https://android.googlesource.com/platform/external/iptables/), we have multiple branches showing different releases - e.g. master, brillo, lollipop, marshmallow, etc. For each of them, there are different versions - marshmallow-cts-release, marshmallow-dev, marshmallow-dr-dev, marshmallow-dr-dragon-release, marshmallow-dr-release, marshmallow-dr1.5-dev, marshmallow-dr1.5-release, marshmallow-dr1.6-release, marshmallow-mr1-dev, marshmallow-mr1-release, marshmallow-mr2-release, marshmallow-mr3-release, marshmallow-release.
How do we identify which branch is implemented in a particular device? Also, there is an option to select tags - which also list various versions of say Android 7.1.1. If I am searching for source code of one phone (e.g. HTC 10), is there a way that I can identify the source code of that particular device?
2. Do the OEMs such as HTC, Samsung, Motorola, Xiaomi, Google etc. use the Android source code in the form that is available at Git repository (open source) or do they make some modifications in the source code for their devices? Further, if there are any modifications, is it possible to view the final source code used by OEMs?
Any help in this regard is highly appreciated.
Thanks in advance!
Regards,
John
I'm not exactly an expert but I'll throw in my two cents.
1. I'm not sure how to find the specific branch that a device uses. But this is Android, and you can pretty much install whatever if it's compatible. If you see this ( http://www.htcdev.com/devcenter/downloads ) and sort by device "HTC 10" you'll see many different results. The type on some look similar to branch names (MR, MR 2, etc) so I assume that's related to it.
2. Most OEMs do modify Android a bit, and you can see the source for any of them. Websites like I linked above (htcdev.com) are the OEM open source packages. If you download the kernel source for the HTC 10 all you would do (usually, some packages are annoying) is extract the contents into your working directory and overwrite existing files. You could probably run diff to view the actual changes they implement.
Hope this somewhat helps!
Edit: To see other OEM open source sites just Google the vendor and open source. Like for LG I Google "LG open source" and it's the first one.
Sent from my LG-H901 using Tapatalk