Re: libinit idea

Hi Bruno,

To preface this, I'd like to say that I do believe that FreeBSD does need a more modern init system.  SMF on Solaris and Launchd on Darwin both have some advantages.  These are what I see as the limitations in our current design (not in priority order):

1) Options are implicit.  Because init scripts are written in a Turing-complete language, including routines scattered across a large number of files, it's very difficult for a tool (e.g. a GUI system administration tool like the SMF GUI on Solaris) to enumerate all of the options.  Additionally, these options are untyped so a GUI that does manage to find them can't tell whether it should be displaying a checkbox (for a boolean value) or a text field.  Additionally, it can't tell what the dependencies are between them.

2) Dependency information between services is poor.  If service A depends on B, and B is not running, then starting A should start B.  If nothing else depends on B and B was not explicitly started, then B should stop.  The last part is the hard one with shell scripts, because it requires maintaining the dependency graph and effectively garbage collecting it (explicitly started services are roots, dependencies are references).

3) It's easy for tools to add lines to rc.conf, it's hard to remove them.  If you're administering a large number of nodes, you want to be able to push out updates to all, in a way that doesn't clobber local changes.  Text file processing here is always going to be a fragile hack.

4) Shell scripts are a good tool for invoking complex sequences of command-line programs, but have a measurable overhead relative to fork() / exec() for running a single command.  Most rc actions just run a single program, we should only be doing a shell invocation for ones that are more complex.

5) In a world where we're moving towards sandboxing services via Capsicum, the service launcher needs to be able to create services with a potentially large set of initial file descriptors (including a socket to Casper), based on the configuration policy.

6) As with launchd, it would make sense for a service management framework to subsume inetd and cron (and devd!) functionality, because timer events, network events, system startup, system shutdown, device attachment, and explicit service starting from the command line are even sources that trigger changes to the service dependency graph.

7) Inspecting the system status is very hard with the current system.  I want to be able to see a list of all running services, along with the reason why they were started.

8) Again, in capsicum world we'd like inetd-like functionality for UNIX domain sockets, so that it's possible to lazily start things like powerboxes.

In terms of design, I don't think that turning rc scripts into libraries is a win.  I would like to see them replaced by a declarative config file in a structured format that provides dependency information, configuration options, and the commands required to start / stop the service (which can be shell scripts, but don't have to invoke a shell).  The configuration options should be separate from the configuration variables, with the former provided by the package and the latter by the system administrator.

We already have, in the base system, a library for parsing a structured configuration file format that is used for the pkg tools and a library with an efficient binary serialisation (libnv) that is used for various Capsicum-related functions.  I'd like to see these combined so that in embedded devices all of the configuration files could be stored in the binary serialisation (which is denser and faster to parse) and turned back into the human-readable one only for editing.  

I do like the idea of an init library, so that the functionality is easily reusable and can be run in nested scopes.  I don't think we gain anything by rewriting shell scripts in C, other than a maintenance headache and a requirement that all port maintainers be fluent C programmers...

David


On 22 Feb 2014, at 23:54, Bruno Lauzé <brunolauze_at_msn.com> wrote:

> https://github.com/brunolauze/libnit
> 
> I know there's really big debate about init system but here's my tentative to propose a new model to replace rc.
> 
> Let's call it libinit but the name as no significance for now.
> 
> I started coding a library with the following architecture.
> 
> the main idea is to rewrite rc in C language.
> 
> a utility called system would act a little bit like service command does.
> 
> a folder would contains libraries instead of scripts inside [target]/etc/rc.d
> so we can add as many librairies a user desire and interlink the order of each piece among all like in rc.
> 
> each library would follow and expose the following pattern:
> 
> char **provide(); /* returns all the PROVIDE a library contains */
> 
> then for each provide() value the library would expose :
> 
> XXX_provide()
> XXX_require()
> XXX_before()
> XXX_keywords()
> 
> and optionally:
> XXX_canstart();
> XXX_prestart();
> XXX_start();
> XXX_status();
> XXX_stop();
> 
> and also:
> 
> XXX_mycommand(int argc, char **argv);
> 
> essentially repeating the rc.subr  model
> 
> system utilty would source /etc/defaults/rc.conf, then source result of rc_conf_files loaded
> 
> On init, /sbin/init would call /sbin/system init instead of running script /etc/rc
> 
> on init, system would scan folder (let's suppose /lib/init.d and /usr/local/init.d for now)
> try dlopen() each *.so* files
> and grab provide(); xxx_provide(), xxx_require(), xxx_before() and xxx_keyword() for each one.
> compile a list of "service" discovered and do an "rcorder".
> 
> The benefits is to avoid firing so many utility to manage to init the system.
> 
> Replicating all small helper function from rc to C language like load_kld would avoid opening a process and do real syscall at moment.
> Heavily use pthread, waitpid, etc...
> 
> So instead of firing /sbin/devfs /dev rule -s 1 applyset 
> call direcly what's would run inside devfs -> rule_main in src/sbin/devfs/rule.c ...
> cut the fat
> 
> here's an example to show /etc/rc.d/abi conversion to abi.c
> 
> abi.h:
> #ifndef __ABI_H__
> #define __ABI_H__
> #include "../default.h"
> 
> #define PROVIDE         abi
> #define REQUIRE         { "archdep" }
> #define KEYWORD         { NOJAIL }
> 
> #include "../common.h"
> 
> #endif
> 
> 
> abi.c:
> #include "abi.h"
> 
> int sysvipc_start()
> {
>         if (load_kld("sysvmsg"))
>                 if (load_kld("sysvsem"))
>                         return load_kld("sysvshm");
>         return -1;
> }
> 
> int linux_start()
> {
>         return load_kld("linux");
> }
> 
> int srv4_start()
> {
>         if (load_kld("svr4elf") == 0)
>                 return load_kld("svr4");
>         return (-1);
> }
> 
> #define __canstart
> int abi_canstart()
> {
>         return is_enabled("sysvipc") || is_enabled("linux") || is_enabled("srv4");
> }
> 
> int abi_start()
> {
>         int err1 = 0, err2 = 0, err3 = 0;
>         if (is_enabled("sysvipc")) err1 = sysvipc_start();
>         if (is_enabled("linux")) err2 = linux_start();
>         if (is_enabled("srv4")) err3 = srv4_start();
>         return err1 && err2 && err3;
> }
> 
> #include "../common.c"
> 
> 
> where common.h and common.c implement everything by default a little bit like rc.subr does.
> e.g: PID_FILE and COMMAND macros implement the start by itself, etc...
> 
> 
> as you can see really similar to what we have in the script file...
> 
> Then the system utility would also allow digging into the libraries with command like:
> system accounting rotatelog
> etc..
> 
> I uploaded a quick start to show some code and expose more the idea.
> 
> https://github.com/brunolauze/libinit
> 
> 
> 
> Thanks in advance for your comments. 		 	   		  
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>