The fragile balance between fast and reliable

June 17, 2010 · Posted in Gentoo, Linux

Being a Distro developer, or a packager if you prefer, is not always that simple. Many people think that our “job” is quite straight forward. All we need to do is to read the INSTALL file and then adapt the instructions into an ebuild format and we are done. Well it is definitely more than that.

There are a number of factors that should be considered before pushing a package into tree. From {C,CXX,LD}FLAGS, use flags, documentation, to multilib support for amd64 platforms, 64bit stability for common 64bit issues such as implicit pointer conversion and many many more. Due to this, we tend to spend many hours digging around the source code, trying to understand how it works, especially when we need to write patches about it. What we usually do is to fix stuff and then send our patches upstream. To be honest I don’t know how other distros work :)

If someone take a look on our bugzilla he will notice many bugs about version bumps of existing packages, feature requests and many more. One may consider that bumping a package is a trivial process. Well it is NOT!

Every time a new version is available, a series of tests need to be run to make sure we retain a high level of QA in our tree. Such tests involve multiple compilations with various compilation flags, compilers, use flags, etc etc

However, the pressure to ship a ‘famous’ package to the portage tree is quite stressful leading to broken ebuilds or badly behaved applications. Whilst I do understand the thirst for bleeding edge packages as soon as possble ( I am still a Gentoo user after all ), I try not to distract from my target for high QA on tree packages.

This is basically a reminder to our users that we ( developers ) are users like them, so we do want new and working packages as soon as possible like they do. We enjoy what we do, we want to do that and we want everything to work as expected :)

Tags: Gentoo

Comments

9 Responses to “The fragile balance between fast and reliable”

RealNC on June 17th, 2010 9:41 pm

What’s an “implicit pointer declaration”? I hear that term for the first time in 15 years as a (hobbyist) C/C++ dev :P
Markos on June 17th, 2010 10:41 pm

I am not sure I can explain it that good.

I think the best code case is the following ( copied from [1] )

sizeof(int) = sizeof(long) = sizeof(pointer)

This relationship is not codified in any C programming standard, but it is valid for the ILP32 data model. However, it is not valid for two of the three 64-bit data models described above, nor is it valid for the LP32 data model.

Moreover, when you cast a function result into a pointer without previously define the function, the compiler complains about that in a way similar to “Dude, i am assuming that the result is going to be an int” which works on 32bit arches ( sizeof(int)=sizeof(pointer) ) but not on amd64 which will certainly lead to segmentation fault :)

My explanation doesn’t cover 100% the issue but I hope I helped you understand what is this about

Read the resources ( especially the third one ) since they explain what is going on in a very detailed way. [2] is just a bug wrt this issue

[1]: http://www.unix.org/whitepapers/64bit.html
[2]: http://bugs.gentoo.org/264884
[3]: http://stackoverflow.com/questions/977233/warning-incompatible-implicit-declaration-of-built-in-function-xyz
RealNC on June 17th, 2010 11:12 pm

That’s called “implicit pointer cast/conversion”, not “declaration” ;-)
Markos on June 17th, 2010 11:14 pm

Ah stupid typo
Jesse on June 18th, 2010 2:26 am

I know this is more of a “process” issue, versus truly “technical” issue, but isn’t this what dividing “stable” vs. “unstable” should be about? In other words, “unstable” would mean “hey, you’re lucky if it works” vs. stable would mean “we’ve run it under this arch, and it appears to work”? Debian has “stable”, “unstable” and “testing”, which might be even more clear to users what risks they are taking with a given package on a given arch, by comparison with Gentoo’s two-tiered package division.
RealNC on June 18th, 2010 2:37 am

Gentoo “unstable” is not “you’re lucky if it works.” Unstable is “it’s supposed to work, we don’t see a reason why it shouldn’t work, but just in case it doesn’t, you should report it.”

Stable is “we and a lot of others tested it and it works.”
Jesse on June 18th, 2010 10:45 am

RealNC, that’s sort of how it is now. I’m saying it should be considered that this isn’t the best way to divide use case, and perhaps looking at other distros and how they divide their package trees might be of value to Gentoo.
Markos on June 18th, 2010 2:08 pm

According to the docs

“The testing branch is exactly what it says – Testing. If a package is in testing, it means that the developers feel that it is functional but has not been thoroughly tested. You could very well be the first to discover a bug in the package in which case you could file a bugreport to let the developers know about it.” [1]

However this is rather abstract. From my point of view, testing it is supposed to work *but* some failures are acceptable. Moreover, if a package is consider rather “dangerous” we mask it in order to prevent ~testing users from merging it. There is a fine line here, but we try to keep testing in a perfect condition ( I consider testing more “stable” than stable branch ).

Stable branch is excellent for servers and/or other situations where you do not require to update your system very often. But for power/desktop users I would strongly recommend testing branch. After all, fixing your system every now and then it is a good way to understand it better :)
Markos on June 18th, 2010 2:08 pm

[1]: http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=3