Re: FYI: what it takes for RAM+swap to build devel/llvm40 with 4 processors or cores and WITH__DEBUG= (powerpc64 example)

From: Brooks Davis <brooks_at_freebsd.org>
Date: Thu, 30 Mar 2017 17:55:19 +0000
On Thu, Mar 30, 2017 at 07:26:19PM +0200, Dimitry Andric wrote:
> On 30 Mar 2017, at 19:06, Alexey Dokuchaev <danfe_at_FreeBSD.org> wrote:
> > 
> > On Mon, Mar 27, 2017 at 11:41:40AM +0200, Dimitry Andric wrote:
> >> On 26 Mar 2017, at 23:36, Mark Millard <markmi_at_dsl-only.net> wrote:
> >>> ...
> >>> Also interesting was:
> >>> 
> >>> Installed packages to be REMOVED:
> >>> 	llvm40-4.0.0.r4
> >>> 
> >>> Number of packages to be removed: 1
> >>> 
> >>> The operation will free 49 GiB.
> >> 
> >> Yes, this is big.  But there is no real need to build the llvm ports
> >> with debug information, unless you want to hack on llvm itself.
> > 
> > Cc'ing jmd_at_ and rezny_at_.
> > 
> > I've been watching increasing size of our LLVM packages with increasing
> > worry.  This is from my tinderbox cache:
> > 
> >  $ % env LANG=C ls -lh llvm3*
> >  -rw-r--r--  1 root  wheel    17M Jan 29  2016 llvm35-3.5.2_1.txz
> >  -rw-r--r--  1 root  wheel    18M Mar  7  2016 llvm36-3.6.2_2.txz
> >  -rw-r--r--  1 root  wheel    27M Feb 28 01:05 llvm37-3.7.1_4.txz
> >  -rw-r--r--  1 root  wheel   207M Jan 19 18:20 llvm38-3.8.1_5.txz
> >  -rw-r--r--  1 root  wheel   244M Mar 23 16:42 llvm39-3.9.1_2.txz
> > 
> > Dimitry, do you know what had causes such a huge bump in 37 -> 38?
> 
> Yes, up to llvm37, the ports were split in devel/llvmXY and lang/clangXY
> parts, with separate ports for e.g. compiler-rt and other LLVM projects.

It's mostly that we build both shared and static libraries.  llvm37
merged clang and llvm (with a clang37 metaport).  Dynamic builds were
broken for a while too which blew up program size.

> > They take lots of time to build and package.  And given that llvm
> > is indirect dependency of any X11-related port, it pessimises their
> > build times as well (devel/libclc now requires devel/llvm40 after
> > r437268).
> 
> The previous split looks pretty hard to maintain, so that is most likely
> the reason for combining all components in one port after 3.8.
> Unfortunately the side effect is that it is way less granular.

I kept it up for several revisions past when it was desupported, but
it's absolutly impossible with the cmake infrastructure unless we want
want to build all of LLVM four times to get clang, llvm, lldb, and lld.
I'm pretty sure that would case more complaints. :)

> If we ever get infrastructure for generating multiple packages out of
> one port, the devel/llvm* ports are very good candidates. :)

Very much so.

> > With 49 GiB llvm40, I guess I won't be able to build-test post as my
> > hardware would just not be capable enough.
> 
> As said, this is because of WITH_DEBUG.  Don't use that for the llvm
> ports, for now.  It will also allow you to build them with much less RAM
> in the machine: especially linking can take multiple GB when debuginfo
> is enabled, and optimization is off.

I'm looking into improving WITH_DEBUG.

-- Brooks

P.S. Somewhat off topice, but related.  FAIR WARNING: the days of
self-hosted 32-bit systems are numbered.  Switching to lld from our
ancient BFD linker will probably buy us some time, but I'd be surprised
if you will be able to build LLVM+CLANG with a 2GB address space in 5
years.  The sooner people make their peace with this, the better.

Received on Thu Mar 30 2017 - 15:55:28 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:11 UTC