Why compile programs yourself? (Linux)

[caruso]

Using precompiled programs from outside the official repositories is dangerous even if the programmer is benevolent; I understand as much. So, how far is it safer to compile a program yourself? I've heard that it also allows you to test experimental features and things like that, but I'm only interested in system integrity and stability right now.

[pav]

Due to the extreme fragmentation of the Linux landscape, it is virtually impossible for the author to provide a binary (executable) download that will work everywhere. Linux people generally choose to sacrifice backwards compatibility, because writing shiny new features is so much more fun than maintaining compatibility shims, and really, open source volunteer programmers prefer to work on what they have most fun working on. Thus, if some subsystem is changed, and some interface or protocol that your program uses to talk to that subsystem changes, it is often impossible to provide executable that would work correctly with both old and new version. This is in stark contract with say, Microsoft, who understand the importance of forward compatibility (and have a required moneypurse ready), where Windows apps written in 1995 will execute correctly on Windows 8 released in 2012. Doing that on Linux is rather impossible, except for most trivial command line programs.

Thus, Linux users are bound to compile for themselves. Source code can be adjusted for changed interfaces, if the user is skilled enough. Binary executable OTOH cannot be modified.

This is where distribution vendors step in, and try (more or less successfully) to package the mess, that Linux world is, into a working and stable system. Since distribution is essentially a list of component versions, it is possible for software vendors to target individual distribution releases even outside distribution provided distribution channel. But that's it.

Well, enough of rant already.

[Derakon]

Using precompiled programs from outside the official repositories is dangerous even if the programmer is benevolent; I understand as much. So, how far is it safer to compile a program yourself? I've heard that it also allows you to test experimental features and things like that, but I'm only interested in system integrity and stability right now.

Safer? It's a question of how trustworthy the source is, just like with precompiled programs. You can download source code that has an exploit in it, just as you can download an executable that has an exploit in it -- compiling the code yourself doesn't prevent the exploit from running. Sure, compile-it-yourself means that you could in theory read through the source code, find the exploit, and remove it yourself, but in practice probably less than one in a hundred thousand people is going to bother with that.

That said, if that one person finds an exploit in the source code, they'll publicize their discovery and it'll get fixed -- there's absolutely community value in the open-source model. But I don't think there's a significant security difference between a program someone you trust compiled for you and a program you compiled yourself. As pav described, the main reason you compile yourself is because there's no guarantee that you'll have the correct DLLs for the precompiled program on your system, so it wouldn't run anyway.

[Mikko Lehtinen]

I've been providing FayAngband executables compiled in Debian Squeeze. They work out of the box in Ubuntu, Knoppix, and other Debian-based distros -- probably the majority of the field? I need to compile two versions, 32-bit and 64-bit. (Of course the source is available too.)

The "proper" way to do this would be to create a .deb package, I guess. I have no experience about that.

I imagine it would get much more complicated if I wanted to provide SDL or other fancy graphical stuff with exotic library dependencies.

[caruso]

But I don't think there's a significant security difference between a program someone you trust compiled for you and a program you compiled yourself.

Okay, that's one of the main issues I was aiming at. So your best bet is relying on the popularity of the project, your malware scanner and the OS updates

As pav described, the main reason you compile yourself is because there's no guarantee that you'll have the correct DLLs for the precompiled program on your system, so it wouldn't run anyway.

I've heard of other cases were the program runs perfectly alright but then unintentionally wrecks your system during the next upgrade, because of screwed up package dependencies or something like that. So, at least that risk can be excluded when you compile a program yourself, right?

I would like to add that I still usually play on Windows; just wanted some more independence from the repositories, Wine and DOSBox in case I totally convert to Ubuntu.

Thanks to all contributors so far

[fph]

I'm only interested in system integrity and stability right now.

Then your best bet is not to compile anything and stick with your distribution's official repositories. They are better tested, and compiled/packaged by competent programmers who know and care about compatibility issues and vulnerabilities.

The only case in which you should not use your distribution's official packages is if you want the new bleeding-edge (and more unstable) version or if you are interested in modifying/hacking/customizing the program.

I've heard of other cases were the program runs perfectly alright but then unintentionally wrecks your system during the next upgrade, because of screwed up package dependencies or something like that. So, at least that risk can be excluded when you compile a program yourself, right?

You have far more chances of screwing up the system and run into dependencies problems if you compile yourself some important part of it, such as the X server. What you heard is either FUD or stories from 15 years ago, when you had to recompile the kernel to get your sound card working (if you were lucky).

I would like to add that I still usually play on Windows; just wanted some more independence from the repositories, Wine and DOSBox in case I totally convert to Ubuntu.

I am afraid you have got your priorities wrong. Nowadays, repositories = tested and works out of the box = good; compiling from source = lots of dependencies and version conflicts to care about; do it only if what you're looking for is not in the repos.

[Therem Harth]

In my (limited) experience, Linux is much less friendly than Windows if you want to run binaries you downloaded from wherever, but much more friendly if you want to tweak things and compile them yourself; the latter, I've found, routinely takes far more time and effort on Windows. IMO Linux is an OS that's really much more geared towards developers, hobbyists, and general geeks than most other stripes of user, for better or for worse.

NB: I find newer versions of Windows very confusing and Byzantine, so please take what I say with a grain of salt!

[Mikko Lehtinen]

Linux live CDs (and virtual machines) are perfect for trying stuff outside repositories. I like live CDs that are Debian-compatible, boot fast, and include everything needed for compiling.

I have a weird problem with newer Windows versions. People are constantly losing their files somewhere in the filesystem and asking my help. "Now where did I save it?" The newer the version, the harder it is to locate the My Documents folder... It's so hard to teach people (especially old people with limited computing experience) to find their My Documents folder that I've given up. Nowadays I just tell them to create their personal folder on the desktop.

[Magnate]

I've been providing FayAngband executables compiled in Debian Squeeze. They work out of the box in Ubuntu, Knoppix, and other Debian-based distros -- probably the majority of the field?

I'm guessing that RPM-based distros are quite a big chunk of the market, but deb-based distros are probably in the top three.

The "proper" way to do this would be to create a .deb package, I guess. I have no experience about that.

Well, it's taken me about three years to get comfortable packaging V (I anticipate that packaging 3.4.0 will take me about a tenth of the effort that it took me to package 3.1.1), so I'm happy to help you short-circuit some of that if you want to learn. Or if you just want it packaged into a deb without learning the process, I could probably do that in about half an hour and send it to you for onward distribution. (I assume that your installation procedure is no more complex than V's 'make install').

Once you have a .deb, you can use the alien package to convert it into an RPM, immediately doubling your market share.

I imagine it would get much more complicated if I wanted to provide SDL or other fancy graphical stuff with exotic library dependencies.

Not really - writing (or converting V's) main-sdl.c is going to be much more work than adding the build-deps.

[Mikko Lehtinen]

Well, it's taken me about three years to get comfortable packaging V (I anticipate that packaging 3.4.0 will take me about a tenth of the effort that it took me to package 3.1.1), so I'm happy to help you short-circuit some of that if you want to learn. Or if you just want it packaged into a deb without learning the process, I could probably do that in about half an hour and send it to you for onward distribution. (I assume that your installation procedure is no more complex than V's 'make install').

Thanks for the offer! I'm planning to start marketing the game for a wider roguelike audience once the next version, Halls of Mist, is "ready". Having a deb and rpm would be helpful at that point. Learning more about Debian is fun, too.

I compile the game with make -f Makefile.std install.

Having .deb would be even more useful if Mist code was under GPL. Sadly, I have no idea how close it is to being GPL. I can ask Eytan of course. But Ey was based on Vanilla Angband 2.9.3, and that codebase may include lots of mysterious non-GPL stuff... I googled and found this on Andrew Doull's Ascii Dreams blog:

This now means that any Angband variant built from version 3.1.0 or later can take advantage of code hosting on Source Forge and other code repositories which restricted free but not open software, as well as the significant protection of the Free Software Foundation should the Angband code be unfairly expropriated elsewhere.

Not really - writing (or converting V's) main-sdl.c is going to be much more work than adding the build-deps.

I'm not really going to spend time on graphics. Mist is just a hobby and my own personal favorite game; nowadays I'm playing more than coding. I want to concentrate on doing fun things. I love ASCII, and working with graphics would feel like work. Furthermore, my codebase is so antiquated that it would feel like reinventing the wheel. (There's one plus, though. Halls of Mist may be the only living variant that compiles happily for DOS...)

The only interface thing that really bothers me is the message line. My playtesting friend is very annoyed at having to press space all the time. He's bugging me to double the line.

[Magnate]

Thanks for the offer! I'm planning to start marketing the game for a wider roguelike audience once the next version, Halls of Mist, is "ready". Having a deb and rpm would be helpful at that point. Learning more about Debian is fun, too.

I compile the game with make -f Makefile.std install.

Ok, so making a deb should be nice and easy - you just use 'make -f Makefile.std' in your debian/rules, and then use debhelper to do the 'install' bits for you (moving stuff into the right places). Start with the new maintainer's guide - it looks long but is actually nicely chunked up.

Having .deb would be even more useful if Mist code was under GPL. Sadly, I have no idea how close it is to being GPL. I can ask Eytan of course. But Ey was based on Vanilla Angband 2.9.3, and that codebase may include lots of mysterious non-GPL stuff...

I'm pretty sure that Eytan put his own code under the GPL (since he contributed to the GPL version of V) - but it should be easy enough to contact him to confirm this. That just leaves any bits hungover from earlier versions - but almost everything had been put under the GPL. The missing people are listed here, along with what they contributed. I don't know how much work it would be to make Halls of Mist GPL-clean, but it might be worth trying before you release a deb/rpm.

I'm not really going to spend time on graphics. Mist is just a hobby and my own personal favorite game; nowadays I'm playing more than coding. I want to concentrate on doing fun things. I love ASCII, and working with graphics would feel like work. Furthermore, my codebase is so antiquated that it would feel like reinventing the wheel.

Don't worry, I wasn't urging you to support SDL, just reassuring you that the work is not in the packaging of it! I think text-only roguelikes are a fine thing, and look forward to trying it.

The only interface thing that really bothers me is the message line. My playtesting friend is very annoyed at having to press space all the time. He's bugging me to double the line.

Surely the solution to that is the easy_more option? If your version doesn't have it, it's only a few lines of code to add in. (Or if you want to get more sophisticated, look at message handling in Un.)

[Mikko Lehtinen]

Thanks a lot for your help.

The combination of GPL and .deb sounds wonderful. It's might be worth the effort for most variants out there!

Surely the solution to that is the easy_more option? If your version doesn't have it, it's only a few lines of code to add in. (Or if you want to get more sophisticated, look at message handling in Un.)

I really like what Un does. For the time being, easy_more is a good idea.

[Magnate]

Thanks a lot for your help.

The combination of GPL and .deb sounds wonderful. It's might be worth the effort for most variants out there!

Good luck with it - feel free to PM me if you get stuck (or drop in to #angband-dev on freenode if you want to talk to someone in real time).

[caruso]

Then your best bet is not to compile anything and stick with your distribution's official repositories. They are better tested, and compiled/packaged by competent programmers who know and care about compatibility issues and vulnerabilities.

Sorry, I meant that when compiling a program, I find it more important to maintain the system integrity and stability (rather than tweak the program). The Ubuntu repositories lack several open source games, and I'd just like to make them work without wrecking my system

What you heard is either FUD or stories from 15 years ago, when you had to recompile the kernel to get your sound card working (if you were lucky).

The article appeared only six years ago, and on a Ubuntu wiki.

Nowadays, [...] compiling from source = lots of dependencies and version conflicts to care about; do it only if what you're looking for is not in the repos.

Even if it's just a game? In that case I better try out live CDs and VMs, like Mikko suggested.

[fph]

Sorry, I meant that when compiling a program, I find it more important to maintain the system integrity and stability (rather than tweak the program). The Ubuntu repositories lack several open source games, and I'd just like to make them work without wrecking my system

Ok, agreed, I get your point. Sometimes you can find games on independent repositories which do little harm to the system, unless they are really obscure ones. But sometimes it is a bet which one will do the least damage to your system between compiling and adding extra repositories which try to replace stuff around.

The article appeared only six years ago, and on a Ubuntu wiki.

Here I am missing something... which article?

Even if it's just a game? In that case I better try out live CDs and VMs, like Mikko suggested.

For most games compiling from source is safe, although you need some practice to parse the error messages and figure out that the solution to your problems is, for instance, installing some packages named libsdl1.2-font-dev.