BitcoinTalk

Linux distribution download

143
Satoshi-only
BitcoinTalk
#1
From:
Odin
Subject:
Linux distribution download
Date:
Hi,

I am a software developer and while I am well versed with building things on Linux I thought I might provide some pointers to making like easier for others.


There is one aspect with distributing a single download package for Linux that is dynamically linked, "You should use the oldest versions of glibc/libstdc++ you can get away with.  i.e. unless you know you need a new feature or a bug fix take is as a given that the libraries at runtime will be newer and contain those fixes."

The problem with linking against the bleeding edge versions of glibc/libstdc++ is that you force the users to have a bleeding edge system also (to run these binaries).  Which should not be a necessary pre-requisite unless you know of specific cause for it to be so.  Such causes should be explained in documentation. Where as if you simply linked on an older system it would work both on that older system and on the bleeding edge system.

If there are areas of C code which you really-really need some new and recent GCC optimization to be enabled, consider providing the dumped out ASM file *.s which you generate once from your bleeding edge GCC version but include in the source to allow anyone to complile (even on older GCC versions).


Here are some recomendations:
 * Use the oldest base distribution you can get away with to build the binaries.  For example the most recent copy of the _PREVIOUS_ release of debian.  Not the most update to date copy of the current release of debian.
 * Provide a download containing static binaries (use the "-static" option to linking and maybe add this so the name of the executables "bitcoind-static"). Provide this in a seperate download file.
 * Make the source distribution build work from working copies of the libraries they need.  In particular boost and wxWidgets, if you know you need very recent versions of these to build against, then make building and linking work right out of their respective build tree (so you don't have to install them).  Installing them in /usr/local maybe an issue (it certainly is for me) but having themin the current working directory just for bitcoin's use would be fine.  At the end of the day all bincoin needs is access to header filer and DSOs or *.a to build and access to the DSOs at runtime.


I can put a 24/7 headless bitcoind up but because of these issues, until I find/make time to resolve them I can not at this time.
BitcoinTalk
#2
From:
Odin
Subject:
Re: Linux distribution download
Date:
Sorry to post to my own post.


One possible solution would be to create a project on the OpenSUSE OBS platform http://wiki.meego.com/Build_Infrastructure .

This platform is an auto-builder anyone can use (by creating your own account and home area).

You can upload your project and *.spec (build descriptor) and it will build your distribution in the format of all the leading Linux distributions (OpenSUSE, CentOS/RHEL, Fedora, Ubuntu, ...).  So there is no need for you to maintain them seperately it will do that for you.

The result will be a *.deb or *.rpm or whatever your platform uses.
BitcoinTalk
#3
From:
satoshi
Subject:
Re: Linux distribution download
Date:
Yeah, acutely aware that I should have stayed on 9.04 or 9.10.  It's a lot more work to downgrade than upgrade and I've been squeezed for time.  Ubuntu is the most popular distro, so I'm staying with that.
BitcoinTalk
#4
From:
Odin
Subject:
Re: Linux distribution download
Date:
Quote from: satoshi on July 29, 2010, 10:17:24 PM
Yeah, acutely aware that I should have stayed on 9.04 or 9.10.  It's a lot more work to downgrade than upgrade and I've been squeezed for time.  Ubuntu is the most popular distro, so I'm staying with that.

Fair enough.  I've not found any version of bitcoin that works with any enterprise version of Linux I have found.  Often enterprise linux uses older tried and trusted versions several years old.  This has parity with the LTS series of Ubuntu.

I think you'd be better doing your distribution with http://releases.ubuntu.com/hardy/  (8.04.4 LTS Hardy Heron).  If you really must use Ubuntu.  This being the _PREVIOUS_ release of their LTS series, once 10.4 LTS passes the 18months old test consider a release engineering rebase.

Maybe you can setup a 2Gb "8.04.4 LTS" disk image using VirtualBox or Xen (just for rolling releases) and keep whatever version you want for desktop/development use.
BitcoinTalk
#5
From:
jgarzik
Subject:
Re: Linux distribution download
Date:
Quote from: Odin on July 29, 2010, 11:10:15 PM
Maybe you can setup a 2Gb "8.04.4 LTS" disk image using VirtualBox or Xen (just for rolling releases) and keep whatever version you want for desktop/development use.

Yeah, some sort of virtual machine like qemu/KVM is really the best way to go, for older distro support.

Constantly reformatting your main dev machine is for the birds Smiley

I have to support OpenSolaris and FreeBSD in my cloud computing project, in addition to the primary target, Linux.  Virtual machines are a life-saver (and cost-saver).
BitcoinTalk
#6
From:
Odin
Subject:
Re: Linux distribution download
Date:
Quote from: jgarzik on July 29, 2010, 11:22:54 PM
Yeah, some sort of virtual machine like qemu/KVM is really the best way to go, for older distro support.

Just to clear up any (potential) misconception, from the use of the phrase "for older distro support".  Just because you compiled your application with an older distro doesn't mean it doesn't work on a newer distro many libraries are backwardly compatible in that way.

When they do change major DSO version (incompatible or breaking changes) some distributions still provide the ability to install the older compat libraries.  Its easier for a new distro to find older compat libraries than it is for an older distro to use a newer GLIBC.


I am somewhat asserting that, if it is indeed possible to compile with older versions (because the features the newer versions add are not actually required by the application) then this version that should be made available via a single binary download (since it will work with the widest audience).


Also to repeat, that the OpenSUSE Build Service is like a remote virtualized service which you can use compile your application on multiple major distributions and the output is a bunch of packages tailored to suit each distro.


The clear problem I see is not with the support for aux lbiraries (Xlib,GTK+,OpenSSL,freetype) but with basic GLIBC and GCC libstdc++.  The problem with this is that you can't expect to simply place the DLLs you need in the same directory as the bitcoin executable and fix up LD_LIBRARY_PATH (man ld.so) so it can see them.

For example lets say that it was compiled with an older GTK+ than is available for your distribution of choice (a DSO major release change, this is indicated in the 2nd "0" in the /usr/lib{,64}/libgtk+-x11-2.0.so.0.1800.9) - note it is still zero, this indicates long term stability re-backward compatibility.  Well it is possible to easily fix this problem without installing the libraries by fixing them up with LD_LIBRARY_PATH.  But for glibc solving the problem in this way is not so straightforward.
BitcoinTalk
#7
From:
Odin
Subject:
Re: Linux distribution download
Date:
Quote from: davidonpda on July 29, 2010, 11:40:06 PM
Make him a virtual box container and upload it or mail him a disk. Then all he needs to do is install vbox and run it.

Google found me http://virtualboxes.org/images/ubuntu/

Maybe head for item 3 for http://downloads.sourceforge.net/virtualboximage/ubuntu-8.04-x86.7z (505Mb download, for a 2.67Gb uncompressed image)
BitcoinTalk
#8
From:
jgarzik
Subject:
Re: Linux distribution download
Date:
Quote from: Odin on July 30, 2010, 12:24:46 AM
Quote from: jgarzik on July 29, 2010, 11:22:54 PM
Yeah, some sort of virtual machine like qemu/KVM is really the best way to go, for older distro support.

Just to clear up any (potential) misconception, from the use of the phrase "for older distro support".  Just because you compiled your application with an older distro doesn't mean it doesn't work on a newer distro many libraries are backwardly compatible in that way.

When they do change major DSO version (incompatible or breaking changes) some distributions still provide the ability to install the older compat libraries.  Its easier for a new distro to find older compat libraries than it is for an older distro to use a newer GLIBC.

Most of what you say is true, but, bitcoin tends to be built against very specific versions of its dependent libs.  Due to that, one tends to build bitcoin against custom compiled libs, regardless of underlying OS version.  That practice makes glibc the primary compatibility worry.
BitcoinTalk
#9
From:
Odin
Subject:
Re: Linux distribution download
Date:
Quote from: jgarzik on July 30, 2010, 02:05:57 AM
Most of what you say is true, but, bitcoin tends to be built against very specific versions of its dependent libs.  Due to that, one tends to build bitcoin against custom compiled libs, regardless of underlying OS version.  That practice makes glibc the primary compatibility worry.


Please cite exactly which library is using which feature of very recent glibc version (or any library for that matter) ?  (just so we can all understand the problem in the way you state).


From what I can see wxWidgets and boost are linked statically.  I guess this is due to the "build against very specific versions of its dependent libs".  But from what I can see there is _NO_ need for either of thes libraries to use a recent glibc, wxWidgets for example only has a check for glibc 2.1 or newer (during its ./configure).  But it is not clear which features of wxWidgets 2.9.0 and/or boost 1.40.0 require the use of a glibc say above version 2.5 (which is also pretty old, but pretty compatibile).  I have sucessfully been able to build these versions (of wxWidgets/boost) against glibc 2.5.

I believe this can be confirmed by the use of -Bstatic in "src/makefile.unix".


The only reason that glibc is a "compatibility worry" (as you put it) is because the project was build on a very new linux system.  This forces all the downloaders to have as-new-a linux system as the build system in order to run it.  This is not because there is some intrinsic compatibility worry as you describe it.


I think I have created a custom wxWidgets 2.9.x build on OpenSUSE build service (OBS).  I shall have a go with "boost" over the new few hours if all goes well.  Then I can commit bitcoind package and get openSuSE builds.  Then I can enable other distribution repos (CentOS, Ubuntu, SLE, Fedora, etc...).

Can anyone confirm if boost "1.40.0" is needed or just "1.40.0 or newer" ?  since 1.42.x is current (and that is already available for a number of platforms as packages).  Can anyone explain the story behind this matter.

With wxWidgets 2.9.x is a development series so it is pre-release, so that is clear cut that a custom version needs to be built to get access to pre-release versions.  But this problem I have solved in an OBS build.
BitcoinTalk
#10
From:
satoshi
Subject:
Re: Linux distribution download
Date:
It can be built with Boost 1.37 or later.