Re: `kldload` doesn't find pf.ko, though `ls` finds it easily enough

On Thu, Nov 18, 2004 at 12:57:35AM +0200, Giorgos Keramidas wrote:
> On 2004-11-17 13:31, Tillman Hodgson <tillman_at_seekingfire.com> wrote:
> > On Wed, Nov 17, 2004 at 10:11:08AM -0800, Mike Hunter wrote:
> > > Check dmesg.  There's a bug somewhere that causes kldload to report "file
> > > not found" to the user even though it was some other error that caused the
> > > problem.  That real error can sometimes be spit out to the console /
> > > syslog / whatever.
> >
> > Oh! I should have checked syslog, my apologies.
> >
> > Nov 17 11:54:41 thoth kernel: link_elf: symbol in6_cksum undefined
> > Nov 17 11:54:41 thoth toor: /etc/rc.d/pf: ERROR: pf module failed to load.
> >
> > My custom kernel doesn't include IPv6. Does pf truly require IPv6?:
> 
> Yes, if you want to load it as a module.  This is exactly the same thing
> I bumped into last night.  Compiling a kernel with INET6 support will
> let you load pf as a module.
> 
> I'm not sure if this is a `bug' that neds fixing though ;-)
> 
I've been investigating this problem recently.  It's a royal PITA
that modules that get compiled with the kernel cannot be linked
into it.  I tried to brute-force it by applying an attached patch
(it has a portion for modules/pf/Makefile), but it soon became
clear that it's not that all simple.  The real fix would involve
carefully looking into every module makefile that generates
opt_*.h files (currently, 63 makefiles), and in the case of a
module build with the kernel (KERNBUILDDIR is defined), generate
only those headers that this "device" is responsible for.

For example, the portion for if_disc in the attached makefile
is correct, as disc(4) can be compiled with any combination of
INET and INET6 options, even when both are undefined.

And the portion for pf/Makefile should look like this, i.e.,
always generate opt_pf.h, but take opt_inet[6].h and opt_bpf.h
from the kernel.

%%%
Index: Makefile
===================================================================
RCS file: /home/ncvs/src/sys/modules/pf/Makefile,v
retrieving revision 1.5
diff -u -r1.5 Makefile
--- Makefile	1 Sep 2004 07:39:12 -0000	1.5
+++ Makefile	18 Nov 2004 08:06:31 -0000
_at__at_ -16,6 +16,7 _at__at_
 	echo "#define DEV_PF 1" > opt_pf.h
 	echo "#define DEV_PFLOG 1" >> opt_pf.h
 
+.if !defined(KERNBUILDDIR)
 opt_inet.h:
 	echo "#define INET 1" > opt_inet.h
 
_at__at_ -28,5 +29,6 _at__at_
 
 opt_bpf.h:
 	echo "#define DEV_BPF 1" > opt_bpf.h
+.endif
 
 .include <bsd.kmod.mk>
%%%

Unfortunately, while it's indeed correct for opt_*.h files,
the SRCS list still assumes that option INET is present in
the kernel (in4_cksum.c is compiled unconditionally), and
some things are just not designed to work without some
options, like option INET in this case, but their dependency
is incorrect.  For example, attempting to build a pf module
with this patch and a kernel without INET option results in:

: In file included from /usr/src/sys/modules/pf/../../contrib/pf/net/if_pflog.c:81:
: ./machine/in_cksum.h:54: warning: "struct ip" declared inside parameter list
: ./machine/in_cksum.h:54: warning: its scope is only this definition or declaration, which is probably not what you want
: ./machine/in_cksum.h:80: warning: "struct ip" declared inside parameter list
: ./machine/in_cksum.h: In function `in_cksum_update':
: ./machine/in_cksum.h:83: error: dereferencing pointer to incomplete type
: ./machine/in_cksum.h:84: error: dereferencing pointer to incomplete type
: *** Error code 1

But if_pflog.c doesn't show a dependency on "option INET"
in sys/conf/files:

# grep if_pflog.c /sys/conf/files
contrib/pf/net/if_pflog.c               optional pflog

I think everyone who has looked into it knows how broken the
current module system is in this respect.

But then we think about opt_bpf.h...  If pf had a module
dependency on bpf, then we could unconditionally compile
BPF support into the pf module.

In the end I gave up on this, as it became clear that
modules build system needs a good redesign.  My current
feeling is that sys/modules should die, and modules
build infrastructure should be generated by config(8).
For example, you create a config file and either tell
it to build everything statically into one kernel object,
or build all (or part) that it can as modules.


Cheers,
-- 
Ruslan Ermilov
ru_at_FreeBSD.org
FreeBSD committer